-
Notifications
You must be signed in to change notification settings - Fork 46
/
ideas-2020.html
584 lines (560 loc) · 30.9 KB
/
ideas-2020.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>OpenWISP GSoC 2020 Ideas List</title>
<link rel="stylesheet" href="../css/reset.css" type="text/css">
<link rel="stylesheet" href="../css/semantic.min.css" type="text/css">
<link rel="stylesheet" href="../css/style.css" type="text/css" media="screen">
<link rel="icon" type="image/x-icon" href="../images/favicon.png" />
</head>
<body class="content">
<div class="ui sidebar vertical menu">
<a class="item" href="../index.html">Home</a>
<a class="item" href="../whatis.html">Features</a>
<a class="item" href="../history.html">History</a>
<a class="item" href="../support.html">Support</a>
<a class="item" href="https://openwisp.io/docs">Docs</a>
<a class="item" href="../thecode.html">Code</a>
<a class="item" href="../about.html">People</a>
</div>
<div class="pusher">
<div class="ui grid" id="top-bar">
<div class="ui computer only column sixteen wide">
<div class="ui center aligned container secondary menu">
<h1 class="item logo">
<a href="../">OpenWISP</a>
</h1>
<div class="right menu">
<a class="item" href="../index.html">Home</a>
<a class="item" href="../whatis.html">Features</a>
<a class="item" href="../history.html">History</a>
<a class="item" href="https://openwisp.io/docs">Docs</a>
<a class="item" href="../support.html">Support</a>
<a class="item" href="../thecode.html">Code</a>
<a class="item" href="../about.html">People</a>
</div>
</div>
</div>
<div class="ui mobile tablet only sixteen wide column">
<div class="ui menu">
<h1 class="item logo">
<a href="../">OpenWISP</a>
</h1>
<div class="right menu">
<div class="menu-open item">
<i class="sidebar icon"></i>
</div>
</div>
</div>
</div>
</div>
<div class="ui grid" id="main">
<article class="ui container">
<h1>Google Summer of Code 2020 Idea List</h1>
<p>
Do you want to apply with us?
We have a page that describes how to increase your chances of success. Please read it carefully.
</p>
<p class="center">
<a class="ui big inverted red button"
href="https://openwisp.io/docs/developer/google-summer-of-code.html">
Read our GSoC Quick Start!
</a>
</p>
<h2>General suggestions and warnings</h2>
<ul class="ui bulleted link list">
<li>
<strong>some details may be missing from the project idea</strong>:
we expect students to do their own research, propose solutions
and be ready to deal with uncertainty and solve challenges
that may come up during the project
</li>
<li>
<strong>
code and prototypes are preferred over detailed documents
and unreliable estimates:
</strong>
rather than wasting your time on writing a very long application document,
we suggest you to invest in writing a prototype (which means the code
may be thrown out entirely) which will help you understand the challenges
of the project you want to work on;
your application should refer to the prototype or other github
contributions you made to OpenWISP that show you have the capability
to succeed in the project idea you are applying for
</li>
<li>
<strong>accepted students will have to create github issues and
a kanban board on github for their project</strong>:
the requirements listed in the idea page will have to be converted
into github issues which shall be tracked from a kanban board on github
to allow the openwisp community to easily keep track of the project progress
</li>
<li>
<strong>students who have shown to have the required skills by contributing to OpenWISP
have a lot more chances of being accepted</strong>.<br>
In order to get started contributing refer to the
<a href="https://openwisp.io/docs/developer/contributing.html">OpenWISP Contributing Guidelines</a>.<br>
Once applicants have completed some basic training, they may start to
work on some aspects of the project they are interested in applying:
all projects listed this year are improvements of existing modules so
it is already possible to start working and complete some of
the tasks listed in the project idea right now.
</li>
</ul>
<h2 id="merging-openwisp-moodules">Merge OpenWISP django modules</h2>
<p><strong>Languages & technologies used:</strong> python, django.</p>
<p>
OpenWISP 2 is composed of several python packages, some of which are
decoupled from the rest of OpenWISP, this has been done for good reasons
which are described in the following section of our documentation:
<a href="https://openwisp.io/docs/general/values.html#software-reusability-means-long-term-sustainability">
OpenWISP values: software reusability means long term sustainability
</a>.
</p>
<p>
After some years in which OpenWISP 2 has been deployed and maintained
we came to the conclusion that in some cases we went a bit too far
with this practice, which has resulted counterproductive in some cases
because it increases maintenance overhead and complexity.
</p>
<p>
The goal of this project idea is to merge some modules which perform
identical operations (although with some distinction) in order to
bring down the complexity of OpenWISP, hence lowering maintenance costs
while keeping the extensibility and reusability of these key modules.
</p>
<p>
These modules are:
</p>
<ul class="ui bulleted link list">
<li>
<a href="https://github.com/openwisp/openwisp-controller">openwisp-controller</a> and
<a href="https://github.com/openwisp/django-netjsonconfig">django-netjsonconfig</a>
</li>
<li>
<a href="https://github.com/openwisp/openwisp-network-topology">openwisp-network-topology</a> and
<a href="https://github.com/openwisp/django-netjsongraph">django-netjsongraph</a>
</li>
<li>
<a href="https://github.com/openwisp/openwisp-radius">openwisp-radius</a> and
<a href="https://github.com/openwisp/django-freeradius">django-freeradius</a>
</li>
<li>
<a href="https://github.com/openwisp/openwisp-ipam">openwisp-ipam</a> and
<a href="https://github.com/openwisp/django-ipam">django-ipam</a>
</li>
</ul>
<p>
Each of this pair performs the same function, the package prefixed with "openwisp-",
for example "openwisp-ipam" extends a base package
(in the case of this example it's django-ipam)
and adds features that are specific to OpenWISP, like multitenancy
(different organization using the same application without interfering with one another).
</p>
<p>
This separation over time is brought more complexity than advantages to the project,
so it's now time to put a stop to it. We just have to make sure each OpenWISP
module is still extensible and reusable so other developers can use it as a base
for their custom applications.
</p>
<p><strong>Measurable outcomes:</strong></p>
<ul class="ui bulleted link list">
<li>Merge django-ipam into openwisp-ipam</li>
<li>Merge django-netjsonconfig into openwisp-controller</li>
<li>Merge django-netjsongraph into openwisp-network-topology</li>
<li>Merge django-freeradius into openwisp-radius</li>
<li>
Migrate the django settings of each module so that the naming is consistent.
For example, all settings of django netjsonconfig are prefixed as
<code>NETJSONCONFIG_[SETTING_NAME]</code> and have to be renamed as
<code>OPENWISP_CONTROLLER_[SETTING_NAME]</code> but backward compatibility
with the old name must be kept in order to facilitate migration:
the old name should be checked first, if a setting with the old naming is
present, the application should use it but also log a warning to
encourage the users to upgrade the name of the setting.
</li>
<li>
Ensure <a href="https://github.com/wq/django-swappable-models">django-swappable-models</a>
are implemented in each merged module and the naming of the swappable settings is consistent.
</li>
<li>
Provide ways to import, extend and override each key area of the application.
Examples:
<a href="https://github.com/openwisp/django-netjsongraph#extending-django-netjsongraph">extending django-netjsongraph</a>,
<a href="https://github.com/openwisp/django-netjsonconfig#id2">Extending django-netjsonconfig</a>.
</li>
<li>
Ensure tests of each merged module are reusable as it's currently done in
django-ipam and django-freeradius
</li>
<li>
Each module shall have has a <code>sample_app</code> in its test suite that extends the main module,
adding some changes on top of it with the sole purpose of testing its extensibility.<br>
In order to better understand this concept take a look at how it's currently implemented in
<a href="https://github.com/openwisp/django-freeradius/search?q=sample_app&unscoped_q=sample_app">django-freeradius</a>. and
<a href="https://github.com/openwisp/django-ipam/search?q=sample_app&unscoped_q=sample_app">django-ipam</a>
(we must port the same concept into the merged modules).
</li>
<li>
Update the documentation of each merged module
(eg: documentation of django-netjsonconfig has to be merged in
openwisp-controller and adapted).
</li>
<li>
Ensure all tests pass and test coverage remains stable.
</li>
<li>
Create a branch on
<a href="https://github.com/openwisp/ansible-openwisp2">ansible-openwisp2</a>
which makes it work with the new merged modules.
</li>
<li>
Create a branch on
<a href="https://github.com/openwisp/docker-openwisp">docker-openwisp</a>
which makes it work with the new merged modules.
</li>
</ul>
<h2 id="openwisp-monitoring">Improving OpenWISP Monitoring towards its first release</h2>
<p><strong>Languages & technologies used:</strong> python, django, timeseries DB, javascript.</p>
<p>
The monitoring module of OpenWISP has not yet been released officially,
but unreleased version is available in a
<a href="https://github.com/openwisp/openwisp-monitoring">github.com repository</a>,
the goal of this project idea is to bring important improvements
to that unreleased module in order to arrive to a first release
and include openwisp-monitoring in the OpenWISP ecosystem.
</p>
<p><strong>Measurable outcomes:</strong></p>
<ul class="ui bulleted link list">
<li>
Implement <a href="https://github.com/wq/django-swappable-models">swappable models</a>.
See <a href="https://github.com/openwisp/django-freeradius/blob/master/django_freeradius/models.py">django-freeradius</a>
as a reference (look for <code>swappable</code> in the code).
</li>
<li>
Abstract the code that communicates with the timeseries DB so that
new timeseries backends can be written to support other timeseries DBs.
</li>
<li>
Add support for another timeseries DB, it could be
<a href="https://prometheus.io/">prometheus</a>,
<a href="http://opentsdb.net/">opentsdb</a> or another good option if there's any.<br>
The decision regarding which one to choose must be backed up by convincing research.
</li>
<li>
Right now the query parameter of a chart (now called <code>Graph</code>)
is editable via the django admin.
This poses several issues: from UX (it's hard to deal with it) to security
(the query could be manipulated to get data to which the user does not have access to).
We need to refactor this part so that the user can choose among a predefined
set of queries, but the list should be customizable using a django setting, so
that users can implement their own queries if needed.
</li>
<li>
The <code>Metric</code> model allows to store different aspects of a metric,
for example, the ping metric allows to store the "reachable" result
(1 for reachable and 0 for unreachable), the packet loss and the
round trip information
(<code>rtt_min</code>, <code>rtt_max</code> and <code>rtt_average</code>).<br>
Each aspect is stored with a different key in the same metric object of the timeseries DB,
but the django model does not show this information and this makes some operations harder
(eg: automatically creating charts which uses some or all the aspects of the metric).<br>
We need to find a solution to this problem, a possible solution would be to introduce
the concept of a metric type and have this kind of general information stored on the metric type,
provided metric types are extensible and users can define their own types
(similarly to the chart query problem listed in the previous point).
</li>
<li>
Rename the model
which is now called <code>Threshold</code> to <code>AlertSettings</code>,
in a backward compatible way
(eg: provide a data migration which creates the alert table first,
copies the data from the threshold table into alert and
then remove the old threshold table, or an equivalently working alternative).
</li>
<li>
Rename the model
which is now called <code>Graph</code> to <code>Chart</code>,
in a backward compatible way
(eg: provide a data migration which creates the chart table first,
copies the data from the graph table into chart and
then remove the old graph table, or an equivalently working alternative).
</li>
<li>
Add a way to disable alerts.
</li>
<li>
Add an "Alert Settings" section in the device admin which
allows to see what alerts are available for a device
and which also allow to modify the threshold or disable/enable the alarm.
</li>
<li>
Add a "Monitoring Checks" section in the device admin which
allows to see what checks are available for a device and add new ones if needed.
</li>
<li>
Add possibility to collect memory (RAM), CPU and flash (disk space) usage
of devices via the API.<br>
The JSON data format supported by the API should follow the
<a href="http://netjson.org/rfc.html#resources2">NetJSON DeviceMonitoring</a> spec.
Automatically create different metrics and charts (one for memory usage, one for CPU usage, one for flash space usage)
whenever this data is received.<br>
Automatically create a related alert setting for each metric which creates an
alert (and sets the device monitoring status to "problem")
if the total value of each metric goes above X,
where X is configurable for each metric with a different setting and defaults to:
95% for RAM, 90% for CPU and 80% for flash usage.
</li>
<li>
Add a check, metric and a relative alert setting
(created automatically for every device, must take into account existing devices by using a data migration)
that is run periodically (by default every 10 minutes).
The check shall find out if the configuration related to the device
has been in the "modified" state for more than
X minutes, where X is configurable with a setting and defaults to 5.<br>
It should also change the monitoring status of the device from "ok" to
either "problem" or "critical", depending on configuration (defaults to "problem").
<br>
The check should not fail but skip if the device has no defined configuration.<br>
Data stored in the timeseries DB which is older than X days can be deleted, where
X is configurable with a django setting and defaults to 2 days.
</li>
<li>
Add a check, metric and a relative alert settings
that will be available only if openwisp-network-topology is in <code>INSTALLED_APPS</code>
(created automatically for every topology, must take into account existing topologies by using a data migration)
that is run periodically (by default every 5 minutes).<br>
The check shall write to the timeseries DB the number of links that are down,
the number of links that are up, the total number of nodes and the number of isolated nodes (nodes without links).<br>
It should then check if the topology has X % or more of its links in "down" state,
in which case it should send an alert. X should be configurable with a django setting and should default to 50%.<br>
Two charts should be created: one which shows the amount of links (one line for the up state and one line for the down state)
and another chart for the amount of nodes (one line for total nodes and one line for isolated nodes).
</li>
<li>Prepare a travis-ci build which runs tests
with both timeseries DB supported,
performs QA checks (like other equivalent openwisp modules)
and collects test coverage data.
</li>
<li>Locate slow tests (tests taking longer than 0.3 secs)
and find out if mocking can be used to optimize them.
</li>
<li>
Add possibility of exporting data with histogram charts.
</li>
<li>Ensure test coverage is above 95%.</li>
<li>
Ensure the README is up to date,
lists all the features available,
provides some screenshots and a detailed explanation of
the usage of the main features added in this project.
</li>
</ul>
<h2 id="openwisp-notifications">OpenWISP Notifications Module</h2>
<p><strong>Languages & technologies used:</strong> python, django, javascript.</p>
<p>
This project idea consists of extracting the notification module
currently implemented in the alpha
<a href="https://github.com/openwisp/openwisp-monitoring">openwisp-monitoring</a>
so that it lives on its own and can be more easily integrated with
other openwisp modules.
</p>
<p><strong>Measurable outcomes:</strong></p>
<ul class="ui bulleted link list">
<li>
Extract the notification module into its own module and
update the code of openwisp-monitoring to make use
of the new module.<br>
At some point early in the project we have to merge this change
into openwisp-monitoring so that the decoupling is completed
as soon as possible.
</li>
<li>
Add the concept of notification type, it may be a list with some default values
which can then be extended by each module.<br>
The first notification types implemented shoud be the ones used
by openwisp-monitoring. This logic shall reside in a sub app
of openwisp-monitoring.
Each notification type shall have a message which is generated from
a configurable text template, the text shall be markdown formatted,
the markdown text shall be expanded into HTML when the notification
is sent via email or viewed via the JS widget.
</li>
<li>
Allow users to disable notifications by notification types,
right now it's only possible to enable or disable the notifications globally.<br>
For each notification type and organization the user should be able
to disable either all notifications or only the email notification.<br>
Example: Receive notifications for organization "default"? <br>
Two checkboxes: Web and Email; if email is checked also web is checked automatically
(because the notification depends on the basic notification being created).<br>
Such a solution would give us the possibility to implement mobile push notifications
in the future by adding a new checkbox for mobile notifications.
</li>
<li>
Prepare a general email notification template which is configurable
(can be changed with a setting) and defaults to using the default
logo of OpenWISP (available in <code>openwisp_utils.admin_theme</code>).
Make the logo configurable and add a way to supply extra CSS if needed.
</li>
<li>
Create a javscript widget for the admin which allows
to view all notification and expand their details by clicking on it.<br>
This widget shall support infinite scrolling
(we have a nice implementation in
<a href="https://github.com/openwisp/django-ipam/blob/master/django_ipam/static/django-ipam/js/subnet.js">django-ipam</a>
which can be reused, either partially or totally) and expand the markdown
into HTML.<br>
The operation of opening a notification detail shall flag it as read.<br>
This feature will require adding API endpoints to get notification list and details.
</li>
<li>
Allow users to disable notifications for specific objects,
either permanently or until a specific date,
with predefined set of durations: permanently,
1 day, 3 days, 1 week, 1 month, custom date.
In the case the disabling is temporary, the setting should be deleted automatically after expiration
by a simple celery task which is run periodically with celery beat.
</li>
<li>
Add celery beat task to automatically delete notifications older than X days,
X defaulting to 90, configurable via setting.
</li>
<li>
Make the notifications module an
optional dependency of
<a href="https://github.com/openwisp/openwisp-controller">openwisp-controller</a>
(by creating a sub app in openwisp-controller)
and create the following notification types in it:
<ul>
<li>new devices registered into the system;</li>
<li>configuration errors and their recovery (has to be triggered when
the configuration status of a device moves into the "errored state" or when it recovers).</li>
</ul>
</li>
<li>
Make the notifications module an optional
dependency of
<a href="https://github.com/openwisp/openwisp-network-topology">openwisp-network-topology</a>
(by creating a sub app in openwisp-network-topology)
and create the following notification types in it:
<ul>
<li>link status changed</li>
<li>new node in the topology</li>
</ul>
</li>
<li>
Prepare a travis-ci build which run tests,
performs QA checks (like other equivalent openwisp modules)
and collects test coverage data.
</li>
<li>Ensure test coverage is above 95%.</li>
<li>
Ensure the README is up to date,
lists all the features available,
provides some screenshots and a detailed explanation of
the usage of the main features of this module.
</li>
</ul>
<h2 id="netengine">Netengine: pull monitoring info from network devices</h2>
<p><strong>Languages & technologies used:</strong> python, NetJSON, SNMP, HTTP, OpenWRT, Linux.</p>
<p>
<a href="https://github.com/openwisp/netengine">Netengine</a>
is a python library that aims to provide a single API
to extract common information from network devices using different protocols
(eg: SNMP, SSH, HTTP) and different firwmares (eg: OpenWRT, AirOS).
</p>
<p>
The library kinda worked but was not brought to production level work
and since OpenWISP 2 is going to have a monitoring system soon, wa want
to rewrite this library to bring its implementation to production level.
</p>
<p>
<strong>NOTE</strong>: backward compatibility does not have to be maintained
we can freely change and improve the API and internal structure of the library
if we think the change is going to improve the quality of the implementation.
</p>
<p>
<strong>WARNING</strong>: to work on this project you need an AirOS device;
OpenWRT can be installed in a virtual box VM, but AirOS cannot (at least
not as easily). If you don't have an AirOS device compatible with the latest
version and you do not indend to buy one you cannot work on this project.<br>
If it ever happens that you get accepted but you fail to buy an AirOS device,
you will be failed.
</p>
<p><strong>Measurable outcomes:</strong></p>
<ul class="ui bulleted link list">
<li>
change the output format of the to_json method to
<a href="http://netjson.org/docs/what.html#devicemonitoring">NetJSON DeviceMonitoring</a>
</li>
<li>
mock all external network calls during tests (SSH, HTTP, SNMP)
</li>
<li>
upgrade the OpenWRT backends to make it work well on the latest two versions of OpenWRT;
if there are incompatibilities, create different backends for each version, ensuring
the shared logic is stored in a common class
</li>
<li>
upgrade the AirOS backends to make it work with the latest 2 version of AirOS;
if there are incompatibilities, create different backends for each version, ensuring
the shared logic is stored in a common class
</li>
<li>achieve 95% test coverage</li>
<li>
improve documentation: at the moment the documentation is really scarce,
we need the documentation to mention all the backends and all the important
features, providing also a few examples of how to use the library
</li>
<li>set up a travis build that performs tests and checks test coverage</li>
<li>implement a check class in <a href="https://github.com/openwisp/openwisp-monitoring">openwisp-monitoring</a>
that makes use of netengine to collect metrics via SNMP</li>
</ul>
<p><strong>NOTE</strong>: this is one of those projects that sounds easy but is not.</p>
<h2>Apply with us</h2>
<p>
Do you want to apply with us?
We have a page that describes how to increase your chances of success. Please read it carefully.
</p>
<p class="center">
<a class="ui big inverted red button"
href="https://openwisp.io/docs/developer/google-summer-of-code.html">
Get started!
</a>
</p>
</article>
</div>
<div class="ui black inverted segment padding-vertical" id="footer">
<div class="ui container stackable two column grid">
<div class="column logo white">
<a href="./">OpenWISP</a>
</div>
<div class="column social">
<a href="https://twitter.com/openwisp" class="ui twitter button">
<i class="twitter icon"></i>
Twitter
</a>
<a href="https://facebook.com/openwisp" class="ui facebook button">
<i class="facebook icon"></i>
Facebook
</a>
<a href="https://www.linkedin.com/company/openwisp" class="ui linkedin button">
<i class="linkedin icon"></i>
Linked In
</a>
<a href="https://github.com/openwisp" class="ui black button">
<i class="github icon"></i> Github
</a>
</div>
<p>© 2008-<script>document.write(new Date().getFullYear())</script> OpenWISP and individual contributors.</p>
</div>
</div>
</div>
<script src="../js/jquery-3.7.1.min.js"></script>
<script src="../js/semantic.min.js"></script>
<script src="../js/scripts.js"></script>
</body>
</html>