logging

Tuesday, January 9, 2024

Introduction to OpenLayers

Introduction

What is 'OpenLayers'?

Openlayers is an open-source JavaScript library for displaying maps and map data in web browsers. It is simmilar to Google Maps, Bing Maps, Leaflet (WMS), Web Feature Service (WFS) protocols and other standards like GeoJSON, GML and many others.

It was developed by a software company MetaCarta ahead of the O'Reilly 'Where 2.0' conference in June 2006, supposedly as an alternative to Google Maps. MetaCarta seems to have disappeared from the internet. This is a screenprint of a still functioning website in 2014:

Installing

Installing build tools for developing OpenLayers based maps on Ubuntu 22.04

Prerequisites on Ubuntu 22.04

Before installing the build tools and sources for developing OpenLayers based mappings on a plain Ubuntu 22.04 system make sure 'curl' and a version of 'node.js' later than version 14 are installed, for instance by running:

sudo apt install curl sudo snap install node --classic --channel=19

Install npm:

curl -0 -L https://npmjs.org/install.sh | sudo sh

* Warning! Be aware that it is risky to pipe a shell script fetched from the internet into a root shell, only do this from a site you trust and on a system that is disposable.* Then install 'git':

sudo apt install git

OpenLayers 'Getting Started'

Now you are able to create the OpenLayers 'getting started app':

npm create ol-app <your app name> cd <your app name>

In order to make the vite development server run on IP address 0.0.0.0 instead of 'localhost' add '--host' to the "start": "vite" line in 'package.json'. And start the development server: ```

npm start ```

If you point your browser to the development server the result should look like:

The Anatomy of OpenLayers 'Getting Started'

Directory structure and files

At this moment the following directories and files will have been created in your application development directory:

myapp/
├── .github
├── .gitignore
├── <u>index.html</u>
├── <u>main.js</u>
├── <u>style.css</u>
├── node_modules
│   ├── .bin
│   .
│   .
│   ├── xml-utils
│   └── zstddec
├── package.json
├── package-lock.json
├── readme.md
├── style.css
└── vite.config.js

The most important files are:

index.html

<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8" /> <link rel="icon" type="image/x-icon" href="https://openlayers.org/favicon.ico" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Using OpenLayers with Vite</title> </head> <body> <div id="map"></div> <script type="module" src="./main.js"></script> </body> </html>

And

main.js

``` import './style.css'; import {Map, View} from 'ol'; import TileLayer from 'ol/layer/Tile'; import OSM from 'ol/source/OSM';

const map = new Map({ target: 'map', layers: [ new TileLayer({ source: new OSM() }) ], view: new View({ center: [0, 0], zoom: 2 }) }); ```

Basic Concepts

The main.js file shows some basic concepts of OpenLayers:

  • Map: from the ol/map module is the core component of OpenLayers. For a map to render, a view, one or more layers, and a target container are needed.
  • View: represents a simple 2D view of the map. This is the object to act upon to change the center, resolution, and rotation of the map. A View has a projection.
  • Source: this function gets (remote) data for a layer.
  • Layer: a map can have multiple layers of different types like Tile, Image or Vector.

Under the hood of OpenLayers 'getting started'

In order to gain some more understanding of the anatomy of the application you can look at the network traffic between your browser and the development server (depends on your browser, but in Firefox: run 'developer tools > network' and 'save as har' then filter the 'url' lines):

          ```
          "url": "http://192.168.50.97:5173/",
          "url": "http://192.168.50.97:5173/@vite/client", ............
          "url": "http://192.168.50.97:5173/main.js",
          "url": "http://192.168.50.97:5173/style.css",
          "url": "http://192.168.50.97:5173/node_modules/.vite/deps/ol.js?v=ef44a998", ........
          .
          "url": "http://192.168.50.97:5173/node_modules/.vite/deps/chunk-ZRN6TD7N.js?v=ef44a998",
          "url": "http://192.168.50.97:5173/node_modules/.vite/deps/chunk-N7KRV6Z5.js?v=ef44a998",
          "url": "http://192.168.50.97:5173/node_modules/.vite/deps/chunk-4TDXP2QV.js?v=ef44a998",
          "url": "https://tile.openstreetmap.org/3 of t/4/4.png",
          "url": "https://tile.openstreetmap.org/3/4/3.png",
          .
          "url": "https://tile.openstreetmap.org/3/5/5.png",
          "url": "https://tile.openstreetmap.org/3/2/5.png",
          "url": "https://openlayers.org/favicon.ico", .............
          "url": "https://tile.openstreetmap.org/3/6/3.png",
          "url": "https://tile.openstreetmap.org/3/6/4.png",
          .
          "url": "https://tile.openstreetmap.org/3/0/5.png",
          "url": "https://tile.openstreetmap.org/3/0/2.png",
          ```

Your browser will load the index.html, main.js javascript and then the openlayer modules followed by the map image which is build of 'png' files which are retreived from the openstreetmap website.

The OpenLayers TileLayer is a map displayed in a web browser by joining dozens of individually requested image (or vector) data files usually 256x256 pixel PNG files. The number of tiles in a map is 2 to the power of the zoom level.

Zoom level 0

Zoom level 1

Zoom level 2

Zoom level 3

(Sample created using tiles.sample.sh)

Thursday, October 19, 2023

IKEA VINDSTYRKA and Raspberry Pi

Introduction


IKEA is a Swedish multinational conglomerate that designs and sells ready-to-assemble furniture, kitchen appliances, decoration, home accessories, and various other goods and home services. Since 2016 IKEA also moved into the 'smart home' business. The smart home offering consists of devices like lamps, speakers, air purifiers, various sensors and an infrastructure of hubs and an app to glue things together.

Here I want to focus on VINDSTYRKA, this is a device to monitor, and report, air quality. My objective is reading information from the device using a Raspberry Pi and store the readings in a database.

The device shows up in the IKEA Home smart app as:
The VINDSTYRKA display shows real time information regarding fine particles or 'particulate matter 2.5' (PM2.5), temperature, humidity and an indication of total volatile organic compounds (tVOC). The information displayed on 'tVOC' is limited to 'arrow up' (rising), 'arrow horizontal' (static) or 'down' (quality improving).
Inside the VINDSTYRKA is a sensor: Sensirion SEN54 sensor. The manufacturers specifications provide more insight into the device capabilities.

VINSTYRKA and IKEA Home smart


Since this sensor is part of IKEA smart home it can be coupled to an IKEA smart home network via a DIRIGERA hub. The communication between the VINDSTYRKA and DIRIGERA hub uses a protocol called Zigbee, the DIRIGERA hub can be accessed via 'ordinary' TCP/IP. This connectivity opens up the possibility of building your own sensor applications. Below is a diagram of the topology now used by me:

Raspberry Pi setup

To gather information from the VINSTYRKA I use a Raspberry Pi running the desktop version of Ubuntu 22.04.3 LTS at the time of writing. Both hardware and operating system choices are not really important , many other combinations may work as well.
On the system as a normal user I setup a directory for the software:
python3 -m venv dirigera
cd dirigera/
source bin/activate
Some explanation:
  • When installing Python packages you should always do this in a virtual environment. My personal prefence is to create create the environment with the name of my project so the prompt shows the name of the active project. In my case:
    jvdmeiden@pi001:~/projects/dirigera$ source bin/activate
    (dirigera) jvdmeiden@pi001:~/projects/dirigera$
    

Then install the dirigera package from https://github.com/Leggin/dirigera:
pip install dirigera
And generate the token for your DIRIGERA using the instructions on https://github.com/Leggin/dirigera:
generate-token <IP address of your DIRIGERA>
Save the generated token in a text file for future use.

Create a database to store information read from the VINDSTYRKA(s)

Create a database for the data from the VINDSTYRKA(s) in your environment (in my case) called 'measurement.db:
sqlite3 measurements.db
Within the database create a table for the measurements:
CREATE TABLE sensor_measurement(timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, room TEXT, temperature INT, humidity  INT,
 pm2_5 INT, tvoc INT);

Create a Python application to read from the VINDSTYRKA(s)

Start out with a script that connects to the hub and reads and prints the sensors information:
import dirigera
hub = dirigera.Hub(
  hub = dirigera.Hub(
  token="..........YOUR..TOKEN.........................................................................",
  ip_address="<IP Address of your DIRIGERA>"
)

current = hub.get_environment_sensors()
for i in range(len(current)):
    print(current[i]);
In my case I have 2 VINDSTYRKA's attached to the DIRIGERA and I get the following output:
EnvironmentSensor(
  device_id='519f42da-bd69-4229-94bf-9c6d1eab4c24_1', 
  custom_name='', 
  room_id='bf0f1f49-7d4b-4116-84d7-78c8c357eac5', 
  room_name='Living room', 
  firmware_version='1.0.11', 
  hardware_version='1', 
  model='VINDSTYRKA', 
  manufacturer='IKEA of Sweden', 
  serial_number='90AB96FFFEF89CA1', 
  can_receive=['customName'], 
  dirigera_client=, 
  is_reachable=True, 
  current_temperature=18, current_rh=75, 
  current_pm25=5, 
  max_measured_pm25=999, 
  min_measured_pm25=0, 
  voc_index=387
)
EnvironmentSensor(
  device_id='dd210409-c149-4e0c-9516-4601cb6b42c8_1', 
  custom_name='Sensor 2', 
  room_id='6dd81fa8-11d9-4a86-8a8a-b465d721df54', 
  room_name='Attick', 
  firmware_version='1.0.11', 
  hardware_version='1', 
  model='VINDSTYRKA', 
  manufacturer='IKEA of Sweden', 
  serial_number='287681FFFE67EC46', 
  can_receive=['customName'], 
  dirigera_client=, 
  is_reachable=True, 
  current_temperature=18, 
  current_rh=70, 
  current_pm25=4, 
  max_measured_pm25=999, 
  min_measured_pm25=0, 
  voc_index=51
)
The information contains the room names I gave the VINDSTARKA's and which data is sent back, the interesting part is:
  • room_name: the name you gave to the VINDSTYRKA
  • current_temperature
  • current_rh (humidity)
  • current_pm25 (PM2.5: fine inhalable particles, with diameters 2.5 micrometers and smaller)
  • max_measured_pm25 and min_measured_pm25=0: the minimal and maximal values of current_pm25
  • voc_index: total volatile organic compounds (tVOC)
It is very interesting that the tvoc information is much more than the simple 'arrow up', 'down' or 'horizontal', shown by the VINDSTYRKA display and the IKEA Home smart app. The information now is an integer in range 1..250. The default value is 100.
To store relevant information in the database create a Python script (I call mine 'sensors.py'):
import dirigera
import sqlite3

conn = sqlite3.connect('<Path to your project directory>/measurements.db')
cursor = conn.cursor()

hub = dirigera.Hub(
  token="..........YOUR..TOKEN.........................................................................",
  ip_address="<IP Address of your DIRIGERA>"
)
current = hub.get_environment_sensors()
for i in range(len(current)):
  cursor.execute('''INSERT INTO sensor_measurement(room,temperature,humidity,pm2_5,tvoc)
    VALUES (?,?,?,?,?)''',
    [current[i].room_name,current[i].current_temperature,current[i].current_rh,current[i].current_pm25,current[i].voc_in
dex])
  conn.commit()
conn.close()
The logic is pretty basic:
  • import the required packages
  • create a hub object
  • connect to the DIRIGERA and get sensor information
  • the returned information is an array of values for each sensor known by the DIRIGERA
  • insert the information for each sensor into the database table


Schedule gathering information using 'cron'

In order to gather information over longer regular intervals create a bash script (sensors.sh) that sets the environment and runs the Python script:
#!/bin/bash
source <Path to your project directory>/bin/activate
python3 <Path to your project directory>/sensors.py
Then create a crontab entry using 'crontab -e':
# m h  dom mon dow   command
  */20  *  *  *  *  <Path to your project directory>/sensors.sh 2>&1 | logger
This will run the script every 20 minutes.
For debugging it is usefull to have the the cronjob write errors to 'logger', the syslog will give a hint of what went wrong in case of problems.

Making sense of the gathered data


After some time the database will contain gathered data like:
sqlite> select * from sensor_measurement where room = 'Living room';
2023-11-19 12:20:02|Living room|18|66|5|59
2023-11-19 12:40:02|Living room|18|66|5|57
2023-11-19 13:00:02|Living room|18|67|5|58
2023-11-19 13:20:02|Living room|18|67|5|61
2023-11-19 13:40:03|Living room|18|67|6|67
2023-11-19 14:00:02|Living room|18|67|6|63
2023-11-19 14:20:02|Living room|18|67|6|58
2023-11-19 14:40:02|Living room|18|67|6|58
2023-11-19 15:00:02|Living room|18|67|7|65
2023-11-19 15:20:02|Living room|18|68|5|168
2023-11-19 15:40:02|Living room|18|75|7|376
2023-11-19 16:00:02|Living room|18|76|8|396
2023-11-19 16:20:02|Living room|18|76|7|360
2023-11-19 16:40:02|Living room|18|75|7|312
2023-11-19 17:00:02|Living room|19|75|6|319
2023-11-19 17:20:02|Living room|19|74|5|103
2023-11-19 17:40:02|Living room|19|76|7|102
2023-11-19 18:00:03|Living room|19|76|7|97
2023-11-19 18:20:02|Living room|19|77|6|108
2023-11-19 18:40:02|Living room|19|76|6|71
2023-11-19 19:00:02|Living room|19|76|5|60
2023-11-19 19:20:02|Living room|19|75|6|43
In this case it shows the increase in tVOC, that coincides with cooking and is not something to get alarmed about.

The peak in tVOC can be plotted as a graph:


Conclusion

Using a DIRIGERA hub and a RASPBERRY Pi it is relativly easy to gather data from the VINDSTYRKA for analysis.

To be continued

The following items are on my to-do list:
  • Gather data from multiple VINDSTYKA sensors and compare them.
  • Correlate the data from the VINDSTYRKA's with weather data.
  • Circumvent the DIRIGERA by connecting at Zigbee level.
  • Replace the VINDSTYRKA by the sensor that is inside Sensirion SN54.
  • Better understand the meaning of various values like tVOC by placing the sensor in a confined space and expose it to things like alcohol vopor, smoke, et cetera.
  • Create a front end that shows the information in graphs.


Links:

Sunday, August 21, 2022

Installing and configuring tinyproxy on Ubuntu 22.04

Install and configure tinyproxy (if you want this system to function as a proxy)

In order to use the system as a proxy install and configure tinyproxy:
sudo apt update
sudo apt upgrade
sudo ufw allow 8888
sudo apt install tinyproxy
sudo touch /var/log/tinyproxy/tinyproxy.log
sudo chown tinyproxy:tinyproxy  /var/log/tinyproxy/tinyproxy.log
Add the IP addresses of systems you want to make use of the proxy to /etc/tinyproxy/tinyproxy.conf:
sudo vi /etc/tinyproxy/tinyproxy.conf
Add lines like:
Allow 198.35.34.96
And restart tinyproxy:
sudo systemctl restart tinyproxy.service
sudo systemctl status tinyproxy.service
Status should look like:
 <user>@<hostname>:~$ sudo systemctl status tinyproxy.service   
 ● tinyproxy.service - Tinyproxy lightweight HTTP Proxy
     Loaded: loaded (/lib/systemd/system/tinyproxy.service; enabled; vendor pre>
     Active: active (running) since Thu 2022-07-14 12:44:52 UTC; 6s ago
       Docs: man:tinyproxy(8)
             man:tinyproxy.conf(5)
    Process: 2523 ExecStart=/usr/bin/tinyproxy $FLAGS (code=exited, status=0/SU>
   Main PID: 2525 (tinyproxy)
      Tasks: 1 (limit: 956)
     Memory: 1.1M
        CPU: 6ms
     CGroup: /system.slice/tinyproxy.service
             └─2525 /usr/bin/tinyproxy

Jul 14 12:44:52 t2202 systemd[1]: Starting Tinyproxy lightweight HTTP Proxy...
Jul 14 12:44:52 t2202 systemd[1]: tinyproxy.service: Can't open PID file /run/tinyproxy/tinyproxy.pid (yet?) after start: Operation not permitted
Jul 14 12:44:52 t2202 systemd[1]: Started Tinyproxy lightweight HTTP Proxy.
(I do not care about the message 'tinyproxy.service: Can't open PID file /run/tinyproxy/tinyproxy.pid' as long as the proxy works...)

Monday, August 1, 2022

Paterson's Worms in Scalable Vector Graphics

Paterson's worms are cellular automata devised in 1971 by Mike Paterson and John Horton Conway.

They model the behavior and feeding patterns of certain prehistoric worms.  These worms fed upon sediment at the bottom of ponds and avoided retracing paths they had already travelled because food would be scarce there.

This behaviour was mathematically modelled by Mike Paterson as described by Michael Beeler (MIT) in 1973.

Ever since I read Martin Gardners feature in Scientific American on Paterson's worms in November 1973, they have fascinated me. For decades I procrastinated writing a turtle graphics program to draw them. I remember making an attempt in Logo long ago... Now I made an implementation in Scalable Vector Graphics (SVG).

The worms crawl a regular pattern along an isometric grid avoiding areas already visited. The grid consists of 'nodes' connected by 'segments', where I use coordinates as follows:



The direction the worm crawls is numbered as follows:



The encoding for worm type I used for the worm type is described by Ben Chaffin and Ed Pegg Jr.. Chaffin created a list of all known types of worms which I incorporated in a drop down menu to select a type.

Nodes I store in an object called field which is actually an associative array (of lines) holding associative arrays (of points on the 'perpendicular' lines). A construct I learned from an article by Peter-Paul Koch. The worm crawls from node to node and line segments are appended to the SVG.

I added colouring: the Worms 'head' will always appear yellow, the end of its 'tail' red.


Some things remain to be done, my wish list:
  • Make a version with possibly multiple worms
  • Increase code efficiency (use typed arrays?)
  • It would be great if you could 'pan' also on the negative side of axes(why doesn't SVG support that...)
  • Add color like here (pworms.wordpress.com)
  • Simple things like adjustable speed and pause
  • ...
Links:

Sunday, February 19, 2017

Francois Morellet 'tirets' in SVG

The french artist François Morellet made a number of paintings called 'Tirets' or 'Dashes' with parallel lines at various angles.

Reconstructed these using SVG (Scalable Vector Graphics).Idea is to be able to run this full screen with the controls optional on mouse over.

Challenge #1 is full screen SVG, can be accomplished using CSS:

<style>
html, body { margin:0; padding:0; overflow:hidden }
svg { position:fixed; top:0; left:0; height:100%; width:100%; z-index: -1 }
</style>





Introduction to OpenLayers

Introduction What is 'OpenLayers'? Openlayers is an open-source JavaScript library for displaying maps and map data in web brow...