This repository contains CSVs in the csv
directory, which are used to define a list of websites to monitor.
The files is parsed on a regular basis (see the workflow file for details) and a notification is sent via Telegram if a change has been detected.
The CSV must have the following structure:
label,active,error_count,url,selector,type,hash
label
: a label or title of the websiteactive
: used to enable or disable this entry, use valuesyes
orno
error_count
: The number of times an error has occured for this entryurl
: the actual URL of the websiteselector
: as CSS selector for elements on the websitetype
: determines the type of the website, usestatic
for static websites ordynamic
for websites, that load most of their contant at runtime. Dynamic websites will be parsed using Selenium. Usestatic
as a default.hash
: The hash of the previous run. Make sure to always provide a value (use a dummy value for new entries)
Example:
label |
active |
error_count |
url |
selector |
type |
hash |
---|---|---|---|---|---|---|
"Thalwil informiert" | yes | 0 | https://www.thalwil.ch/aktuellesinformationen | #informationList | static | db60b21849b715eb4c12d75f285d460de6dfbc17b9429f8f0bfcc78fca76cb2e |
A small script to monitor the contents of a website and get a notification if there are changes.
- Clone this repository
- Run the
setup.sh
script.sh or install manually the dependencies:
source env/bin/activate
pip install -r requirements.py
$ python monitor.py --help
Usage:
monitor.py --url <url-of-website> [--selector <css-selector>] [--wait <number-of-seconds>] [--verbose] [--no-verify]
monitor.py (-h | --help)
monitor.py --version
Options:
-h, --help Show this screen.
--version Show version.
-u, --url <url-of-website> URL of the website to monitor.
-s, --selector <css-selector> CSS selector to check for changes [default: body].
-w, --wait <number-of-seconds> Number of seconds to wait until the URL is checked again [default: 30].
--verbose Option to enable more verbose output.
--no-verify Option to disable SSL verification for requests.
Watch «Amtliche Sammlung» of the City of Zurich:
python monitor.py --url https://www.stadt-zuerich.ch/portal/de/index/politik_u_recht/amtliche_sammlung.html --selector .mod_newsteaser
Ignore SSL certificate errors:
python monitor.py --url https://metaodi.ch --selector div.content --no-verify
Verbose output:
python monitor.py -u https://www.wikidata.org/wiki/Special:Random --selector span.wikibase-title-label --verbose
Check every 5 seconds:
python monitor.py -u https://www.wikidata.org/wiki/Special:Random --selector span.wikibase-title-label --wait 5