woensdag 5 februari 2020

Getting started with Python

What version are you running? In order to find out start a Python Shell and type:
$ python
Python 3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print (sys.version)
3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]

A first sample from 'dive into Python 3':

$ cat sample.001.py 
SUFFIXES = {1000: ['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'],
    1024: ['KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB', 'ZiB', 'YiB']}
def approximate_size(size, a_kilobyte_is_1024_bytes=True):
  '''Convert a file size to human-readable form.

  Keyword arguments:
  size -- file size in bytes
  a_kilobyte_is_1024_bytes -- if True (default), use multiples of 1024
                              if False, use multiples of 1000

  Returns: string

  if size < 0:
    raise ValueError('number must be non-negative')
  multiple = 1024 if a_kilobyte_is_1024_bytes else 1000
  for suffix in SUFFIXES[multiple]:
    size /= multiple
    if size < multiple:
      return '{0:.1f} {1}'.format(size, suffix)

  raise ValueError('number too large')

if __name__ == '__main__':
  print(approximate_size(1000000000000, False))
Produces output:
$ python sample.001.py
1.0 TB
931.3 GiB

zondag 20 oktober 2019

Flask 'Hello world'

This is a description of my tour to get started with Flask.

A Hello world app

Create a directory and a Python venv and install Flask:

mkdir flask
cd flask/
python3 -m venv venv
source venv/bin/activate
pip install flask

Now is a moment for reflection: what did 'pip install flask' install?

$ pip freeze

In this directory create 'hello_flask.py'. That contains as a start:

from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!'

from flask import Flask
-- make Flask available
app = Flask(__name__)
-- create an instance of the Flask class
-- define that the instance is mapped to the home ‘/’ URL
def hello_world()
-- define a function in the class
return 'Hello, World!'
-- define the output of the function

And then run:

export FLASK_APP=hello_flask.py
flask run --host=

And a web server is running:

That results in the following directory structure, mainly existing of the venv:

. . ├── hello_flask.py ├── __pycache__ │   └── hello_flask.cpython-36.pyc └── venv ├── bin │   ├── activate │   ├── activate.csh │   ├── activate.fish │   ├── easy_install │   ├── easy_install-3.6 │   ├── flask │   ├── pip │   ├── pip3 │   ├── pip3.6 │   ├── python -> python3 │   └── python3 -> /usr/bin/python3 ├── include ├── lib │   └── python3.6 │   └── site-packages │   ├── click │   ├── Click-7.0.dist-info │   ├── easy_install.py │   ├── flask │   ├── Flask-1.1.1.dist-info │   ├── itsdangerous │   ├── itsdangerous-1.1.0.dist-info │   ├── jinja2 │   ├── Jinja2-2.10.3.dist-info │   ├── markupsafe │   ├── MarkupSafe-1.1.1.dist-info │   ├── pip │   ├── pip-9.0.1.dist-info │   ├── pkg_resources │   ├── pkg_resources-0.0.0.dist-info │   ├── __pycache__ │   │   └── easy_install.cpython-36.pyc │   ├── setuptools │   ├── setuptools-39.0.1.dist-info │   ├── werkzeug │   └── Werkzeug-0.16.0.dist-info ├── lib64 -> lib ├── pyvenv.cfg └── share └── python-wheels

..'pyc' files... ".pyc files are created by the Python interpreter when a .py file is imported. They contain the "compiled bytecode" of the imported module/program so that the "translation" from source code to bytecode (which only needs to be done once) can be skipped on subsequent imports if the .pyc is newer than the corresponding .py file, thus speeding startup a little. But it's still interpreted. Once the *.pyc file is generated, there is no need of *.py file, unless you edit it."

And the 'wheel files': Wheels are the new standard of Python distribution. Under the hood they are 'zip' files.

It is possible to import the Flask app in a Python shell and see which URL's it maps:

(venv) ~/projects/flask$ cat hello_flask.py from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!' (venv) ~/projects/flask$ (venv) ~/projects/flask$ python Python 3.6.8 (default, Oct 7 2019, 12:59:55) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from hello_flask import app >>> app.url_map Map([<rule '/' (OPTIONS, GET, HEAD) -> hello_world>, <rule '/static/<filename>' (OPTIONS, GET, HEAD) -> static>]) >>> (venv) ~/projects/flask$

Any request to this Flask App that does not match '/' will cause this Flask App to respond with a '404 Not Found':


vrijdag 4 oktober 2019

Raspberry Pi 4 mount / on a SSD

Some performance background

Playing around with Raspberry Pi 4 I ran some benchmarks first to get a feeling of performance:

pi@raspjvdm01:~ $ sudo apt install sysbench

pi@raspjvdm01:~ $ sysbench --test=cpu run

Resulted in:

Threads fairness:
events (avg/stddev):           10000.0000/0.00
execution time (avg/stddev):   92.7503/0.00

CPU is not that fast, the same test on a Digital Ocean droplet ends in:

Threads fairness:
events (avg/stddev): 10194.0000/0.00
execution time (avg/stddev): 9.9943/0.00

Memory test did not run on the Raspberry for some reason.

Now for disk and SD on the Raspberry, first SSD:

pi@raspjvdm01:/data/tmp $ sysbench --test=fileio --file-test-mode=seqwr run

128 files, 16Mb each
2Gb total file size
Block size 16Kb
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential write (creation) test
Threads started!

Operations performed:  0 Read, 131072 Write, 128 Other = 131200 Total
Read 0b  Written 2Gb  Total transferred 2Gb  (176.74Mb/sec)
11311.48 Requests/sec executed

Test execution summary:
total time:                          11.5875s
total number of events:              131072
total time taken by event execution: 10.6246
per-request statistics:
min:                                  0.05ms
avg:                                  0.08ms
max:                                 23.85ms
approx.  95 percentile:               0.07ms

Threads fairness:
events (avg/stddev):           131072.0000/0.00
execution time (avg/stddev):   10.6246/0.00

The same on SD:

Threads fairness:
events (avg/stddev):           131072.0000/0.00
execution time (avg/stddev):   201.1742/0.00

So do not expect performance from SD!

Again compare to a droplet:

Threads fairness:
events (avg/stddev): 411818.0000/0.00
execution time (avg/stddev): 9.8016/0.00

The Raspberry Pi with SSD does perfect!

Setting up the Raspberry to use the SSD as '/'

How I set up and configured my Raspberry Pi to use a USB attached SSD drive:

- Download the Lite Raspbian image (https://downloads.raspberrypi.org/raspbian_lite_latest)
- Create an SD in Ubuntu with 'Startup Disk Creator'
- Insert the SD
- Attach a display, keyboard and network to the Raspberry Pi, and power on (attach power)
- Logon as 'pi' with the default password 'raspberry'
- Run: 'sudo raspi-config' and:
- - Configure the network (I use dhcp with a fixed IP number from my router)
- - Configure a user pi password and/or ssh keys
- - Start sshd

Then I moved the / filesystem to a USB mounted

Create a new folder called /media/newdrive and mounted my external drive (/dev/sda1) and copied /:

sudo mkdir /media/newdrive
sudo mount /dev/sda1 /media/newdrive
sudo rsync -avx / /media/newdrive

Then changed '/boot/cmdline.txt':

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=6e0733c3-02 rootfstype=ext4 elevator=deadline fsck.repair=yes root=/dev/sda1 rootfstype=ext4 rootwait

zondag 19 februari 2017

Francois Morellet 'tirets' in SVG

The french artist Fran├žois Morellet made a number of paintings called 'Tirets' or 'Dashes' with parallel lines at various angles.

Reconstructed these using SVG (Scalable Vector Graphics).Idea is to be able to run this full screen with the controls optional on mouse over.

Challenge #1 is full screen SVG, can be accomplished using CSS:

html, body { margin:0; padding:0; overflow:hidden }
svg { position:fixed; top:0; left:0; height:100%; width:100%; z-index: -1 }

zondag 23 maart 2014

Paterson's Worms in Scalable Vector Graphics

Paterson's worms are cellular automata devised in 1971 by Mike Paterson and John Horton Conway.

They model the behavior and feeding patterns of certain prehistoric worms.  These worms fed upon sediment at the bottom of ponds and avoided retracing paths they had already travelled because food would be scarce there.

This behaviour was mathematically modelled by Mike Paterson as described by Michael Beeler (MIT) in 1973.

Ever since I read Martin Gardners feature in Scientific American on Paterson's worms in November 1973, they have fascinated me. For decades I procrastinated writing a turtle graphics program to draw them. I remember making an attempt in Logo long ago... Now I made an implementation in Scalable Vector Graphics (SVG).

The worms crawl a regular pattern along an isometric grid avoiding areas already visited. The grid consists of 'nodes' connected by 'segments', where I use coordinates as follows:

The direction the worm crawls is numbered as follows:

The encoding for worm type I used for the worm type is described by Ben Chaffin and Ed Pegg Jr.. Chaffin created a list of all known types of worms which I incorporated in a drop down menu to select a type.

Nodes I store in an object called field which is actually an associative array (of lines) holding associative arrays (of points on the 'perpendicular' lines). A construct I learned from an article by Peter-Paul Koch. The worm crawls from node to node and line segments are appended to the SVG.

I added colouring: the Worms 'head' will always appear yellow, the end of its 'tail' red.

Some things remain to be done, my wish list:
  • Make a version with possibly multiple worms
  • Increase code efficiency (use typed arrays?)
  • It would be great if you could 'pan' also on the negative side of axes(why doesn't SVG support that...)
  • Add color like here (pworms.wordpress.com)
  • Simple things like adjustable speed and pause
  • ...