merged with upstream master

README.md

@@ -50,6 +50,13 @@ git clone https://github.com/neuralet/smart-social-distancing.git
 cd smart-social-distancing
 ```
 
+After that, `checkout` to the latest release:
+```bash
+git fetch --tags
+# Checkout to the latest release tag
+git checkout $(git tag | tail -1)
+```
+
 ### Download a sample video
 If you don't have any camera to test the solution you can use any video as an input source. You can download an example with the following command.
 
@@ -239,20 +246,19 @@ All the configurations are grouped in *sections* and some of them can vary depen
   - `Resolution`: Specifies the image resolution that the whole processor will use. If you are using a single camera we recommend using that resolution.
   - `Encoder`: Specifies the video encoder used by the processing pipeline.
   - `MaxProcesses`: Defines the number of processes executed in the processor. If you are using multiple cameras per processor we recommend increasing this number.
-  - `ScreenshotPeriod`: Defines a time period (expressed in minutes) to take a screenshot of all the cameras and store them in S3. If you set the value to 0, no screenshots will be taken.
-  - `ScreenshotS3Bucket`: Configures the S3 Bucket used to store the screenshot.
   - `ScreenshotsDirectory`: Configures the folder dedicated to storing all the images (for example the heatmap reports) generated by the processor. We recommend to set this folder to a mounted directory (such as */repo/data/processor/static/screenshots*)
   - `DashboardURL`: Sets the url where the frontend is running. Unless you are using a custom domain, you should keep this value as https://beta.lanthorn.ai/.
   - `SlackChannel`: Configures the slack channel used by the notifications. The chosen slack channel must exist in the configured workspace.
   - `OccupancyAlertsMinInterval`:  Sets the desired interval (in seconds) between occupancy alerts.
   - `MaxThreadRestarts`: Defines the number of restarts allowed per thread.
+  - `HeatmapResolution`: Sets the resolution used by the heatmap report.
 
 - `[Api]`
   - `Host`: Configures the host IP of the processor's API (inside docker). We recommend don't change that value and keep it as *0.0.0.0*.
   - `Post`: Configures the port of the processor's API (inside docker). Take care that if you change the default value (*8000*) you will need to change the startup command to expose the configured endpoint.
   - `SSLEnabled`: A boolean parameter to enable/disable https/ssl in the API. We recommend setting this value in *True*. 
   - `SSLCertificateFile`: Specifies the location of the SSL certificate (required when you have *SSL enabled*). If you generate it following the steps defined in this Readme you should put */repo/certs/<your_ip>.crt*
-  - [`SSLKeyFile`]: Specifies the location of the SSL key file (required when you have *SSL enabled*). If you generate it following the steps defined in this Readme you should put */repo/certs/<your_ip>.crt*
+  - [`SSLKeyFile`]: Specifies the location of the SSL key file (required when you have *SSL enabled*). If you generate it following the steps defined in this Readme you should put */repo/certs/<your_ip>.key*
 
 - `[Core]`:
   - `Host`: Sets the host IP of the *QueueManager* (inside docker).
@@ -288,7 +294,7 @@ All the configurations are grouped in *sections* and some of them can vary depen
   - `DistMethod`: Configures the chosen distance method used by the processor to detect the violations. There are three different values: CalibratedDistance, CenterPointsDistance and FourCornerPointsDistance. If you want to use *CalibratedDistance* you will need to calibrate the camera from the [UI](https://beta.lanthorn.ai).
 
 - `[Detector]`:
-  - `Device`: Specifies the device. The available values are *Jetson*, *EdgeTPU*, *Dummy*, *x86*
+  - `Device`: Specifies the device. The available values are *Jetson*, *EdgeTPU*, *Dummy*, *x86*, *x86-gpu*
   - `Name`: Defines the detector's models used by the processor. The models available varies from device to device. Information about the supported models are specified in a comment in the corresponding *config-<device>.ini* file.
   - `ImageSize`: Configures the moedel input size. When the image has a different resolution, it is resized to fit the model ones. The available values of this parameter depends on the model chosen.
   - `ModelPath`: Some of the supported models allow you to overwrite the default one. For example, if you have a specific model trained for your scenario you can use it.
@@ -299,27 +305,52 @@ All the configurations are grouped in *sections* and some of them can vary depen
 - `[Classifier]`:
 
   Some of the supported devices includes the *facemask detection* feature. If you want to include this feature, you need to specify the classifier section.
+  - `Device`: Specifies the device. The available values are *Jetson*, *EdgeTPU*, *Dummy*, *x86*, *x86-gpu*
   - `Name`: Name of the facemask classifier used.
   - `ImageSize`: Configures the moedel input size. When the image has a different resolution, it is resized to fit the model ones. The available values of this parameter depends on the model chosen.
   - `ModelPath`: The same behavior as in the section `Detector`.
   - `MinScore`: Defines the facemask detection threshold. Any facemask detected by the model with a score less than the threshold will be ignored.
 
-
-- `[PostProcessor]`:
+- `[Tracker]`:
+  - `Name`: Name of the tracker used.
   - `MaxLost`: Defines the number of frames that an object should disappear to be considered as lost.
   - `TrackerIOUThreshold`: Configures the threshold of IoU to consider boxes at two frames as referring to the same object at IoU tracker.
-  - `NMSThreshold`: Configures the threshold of minimum IoU to detect two boxes as referring to the same object.
-  - `DefaultDistMethod`: Defines the default distance algorithm for the cameras without *DistMethod* configuration.
-  - `DistThreshold`: Configures the distance threshold for the *social distancing violations*
-  - `Anonymize`: A boolean parameter to enable/disable anonymization of faces in videos and screenshots.
-
-- `[Logger]`:
-  - `Name`: Configures the chosen logger. For now, we only support the *csv_logger*.
-  - `TimeInterval`: Sets the desired logging interval for objects detections and violations.
-  - `LogDirectory`: Defines the location where the generated files will be stored.
-  - `EnableReports`: A boolean parameter to enable/disable the reports generation.
-  - `HeatmapResolution`: Sets the resolution used by the heatmap report.
-  - `WebHooksEndpoint`: Configures an endpoint to receive in real-time the objects detections and violations.
+
+- `[SourcePostProcessor_N]`:
+
+  In the config files, we use the *SourcePostProcessor* sections to specify additional processing steps after running the detector and face mask classifier (if available) on the video sources. We support 3 different ones (identified by the field *Name*) that you enable/disable uncommenting/commenting them or with the *Enabled* flag.
+    - `objects_filtering`: Used to remove invalid objects (duplicates or large).
+      - `NMSThreshold`: Configures the threshold of minimum IoU to detect two boxes as referring to the same object.
+    - `social_distance`: Used to measure the distance between objects and detect social distancing violations.
+      - `DefaultDistMethod`: Defines the default distance algorithm for the cameras without *DistMethod* configuration.
+      - `DistThreshold`: Configures the distance threshold for the *social distancing violations*
+    - `anonymizer`: A step used to enable anonymization of faces in videos and screenshots.
+
+- `[SourceLogger_N]`:
+
+  Similar to the section *SourcePostProcessor_N*, we support multiple loggers (right now 4) that you enable/disable uncommenting/commenting them or with the *Enabled* flag.
+  - `video_logger`: Generates a video stream with the processing results. It is a useful logger to monitor in real-time your sources.
+  - `s3_logger`: Stores a screenshot of all the cameras in a S3 bucket.
+    - `ScreenshotPeriod`: Defines a time period (expressed in minutes) to take a screenshot of all the cameras and store them in S3. If you set the value to 0, no screenshots will be taken.
+    - `ScreenshotS3Bucket`: Configures the S3 Bucket used to store the screenshot.
+  - `file_system_logger`: Stores the processed data in a folder inside the processor.
+    - `TimeInterval`: Sets the desired logging interval for objects detections and violations.
+    - `LogDirectory`: Defines the location where the generated files will be stored.
+    - `ScreenshotPeriod`: Defines a time period (expressed in minutes) to take a screenshot of all the cameras and store them. If you set the value to 0, no screenshots will be taken.
+  - `web_hook_logger`: Allows you to configure an external endpoint to receive in real-time the object detections and violations.
+    - `Endpoint`: Configures an endpoint url.
+
+- `[AreaLogger_N]`:
+
+  Similar to the section *SourceLogger_N* (for areas instead of cameras), we support multiple loggers (right now only 1, but we plan to include new ones in the future) that you enable/disable uncommenting/commenting them or with the *Enabled* flag.
+  - `file_system_logger`: Stores the occupancy data in a folder inside the processor.
+    - `LogDirectory`: Defines the location where the generated files will be stored.
+
+- `[PeriodicTask_N]`:
+
+  The processor also supports the execution of periodic tasks to generate reports, accumulate metrics, etc. For now, we support only the *metrics* task. You can enable/disable that functionality uncommenting/commenting the section or with the *Enabled* flag.
+  - `metrics`: Generates different reports (hourly, daily and live) with information about the social distancing infractions, facemask usage and occupancy in your cameras and areas. You need to have it enabled to see data in the [UI](https://beta.lanthorn.ai) dashboard or use the `/metrics` endpoints.
+      - `LiveInterval`: Expressed in minutes. Defines the time interval desired to generate live information.
 
 ### API usage
 After you run the processor on your node, you can use the exposed API to control the Processor's Core, where all the process is getting done.
@@ -328,7 +359,18 @@ The available endpoints are grouped in the following subapis:
 - `/config`: provides a pair of endpoint to retrieve and overwrite the current configuration file.
 - `/cameras`: provides endpoints to execute all the CRUD operations required by cameras. These endpoints are very useful to edit the camera's configuration without restarting the docker process. Additionally, this subapi exposes the calibration endpoints.
 - `/areas`: provides endpoints to execute all the CRUD operations required by areas.
-- `/reports`: a set of endpoints to retrieve the data generated by the processor.
+- `/app`: provides endpoints to retrieve and update the `App` section in the configuration file.
+- `/api`: provides endpoints to retrieve the `API` section in the configuration file.
+- `/core`: provides endpoints to retrieve and update the `CORE` section in the configuration file.
+- `/detector`: provides endpoints to retrieve and update the `Detector` section in the configuration file.
+- `/classifier`: provides endpoints to retrieve and update the `Classifier` section in the configuration file.
+- `/tracker`: provides endpoints to retrieve and update the `Tracker` section in the configuration file.
+- `/source_post_processors`: provides endpoints to retrieve and update the `SourcePostProcessor_N` sections in the configuration file. You can use that endpoint to enable/disable a post processor step, change a parameter, etc.
+- `/source_loggers`: provides endpoints to retrieve and update the `SourceLoggers_N` sections in the configuration file. You can use that endpoint to enable/disable a logger, change a parameter, etc.
+- `/area_loggers`: provides endpoints to retrieve and update the `AreaLoggers_N` sections in the configuration file. You can use that endpoint to enable/disable a post processor step, change a parameter, etc.
+- `/periodict_tasks`: provides endpoints to retrieve and update the `PeriodicTask_N` sections in the configuration file. You can use that endpoint to enable/disable the metrics generation.
+- `/metrics`: a set of endpoints to retrieve the data generated by the metrics periodic task.
+- `/export`: a set of endpoints to export (in csv and zip format) all the data generated by the processor.
 - `/slack`: a set of endpoints required to configure Slack correctly in the processor. We recommend to use these endpoints from the [UI](https://beta.lanthorn.ai) instead of calling them directly.
 
  Additionally, the API exposes 2 endpoints to stop/start the video processing

api/models/api.py

@@ -0,0 +1,12 @@
+from pydantic import Field
+from typing import Optional
+
+from .base import SnakeModel
+
+
+class ApiDTO(SnakeModel):
+    host: str = Field("0.0.0.0")
+    port: int = Field(8000)
+    SSLEnabled: Optional[bool] = Field(False)
+    SSLCertificateFile: Optional[str] = Field("", example="/repo/certs/0_0_0_0.crt")
+    SSLKeyFile: Optional[str] = Field("", example="/repo/certs/0_0_0_0.key")

api/models/app.py

@@ -0,0 +1,21 @@
+from pydantic import Field
+from typing import Optional
+
+from .base import SnakeModel
+
+
+class AppDTO(SnakeModel):
+    hasBeenConfigured: bool = Field(False)
+    resolution: str = Field("640,480")
+    encoder: str = Field("videoconvert ! video/x-raw,format=I420 ! x264enc speed-preset=ultrafast")
+    maxProcesses: int = Field(1)
+    dashboardURL: str = Field("http://0.0.0.0:8000")
+    screenshotsDirectory: str = Field("", example="/repo/data/processor/static/screenshots")
+    slackChannel: Optional[str] = Field("", example="lanthorn-notifications")
+    occupancyAlertsMinInterval: int = Field(0, example=180)
+    maxThreadRestarts: int = Field(5)
+    globalReportingEmails: Optional[str] = Field("", example="email@email,email2@email")
+    globalReportTime: str = Field("06:00")
+    dailyGlobalReport: bool = Field(False)
+    weeklyGlobalReport: bool = Field(False)
+    heatmapResolution = Field("150,150")

api/models/area.py

@@ -0,0 +1,16 @@
+from pydantic import Field
+from typing import List, Optional
+
+from .base import EntityConfigDTO, NotificationConfig, SnakeModel
+
+
+class AreaNotificationConfig(NotificationConfig):
+    occupancyThreshold: Optional[int] = Field(0, example=300)
+
+
+class AreaConfigDTO(EntityConfigDTO, AreaNotificationConfig):
+    cameras: Optional[str] = Field("", example='cam0,cam1')
+
+
+class AreasListDTO(SnakeModel):
+    areas: List[AreaConfigDTO]

api/models/area_logger.py

@@ -0,0 +1,31 @@
+from pydantic import Field, validator
+from typing import List, Optional
+
+from .base import OptionalSectionConfig, SnakeModel
+
+
+class AreaLoggerDTO(OptionalSectionConfig):
+    logDirectory: Optional[str] = Field(example="/repo/data/processor/static/data/areas")
+
+    @validator("name")
+    def validate_name(cls, value):
+        if value != "file_system_logger":
+            raise ValueError(f"Not supported logger named: {value}")
+        return value
+
+
+class FileSystemLoggerDTO(OptionalSectionConfig):
+    logDirectory: str = Field("/repo/data/processor/static/data/sources",
+                              example="/repo/data/processor/static/data/areas")
+
+
+class AreaLoggerListDTO(SnakeModel):
+    areasLoggers: List[AreaLoggerDTO]
+
+
+def validate_logger(logger: AreaLoggerDTO):
+    logger_model = None
+    if logger.name == "file_system_logger":
+        logger_model = FileSystemLoggerDTO
+    # Validate that the specific logger's fields are correctly set
+    logger_model(**logger.dict())

api/models/base.py

@@ -0,0 +1,32 @@
+from pydantic import BaseModel, Field
+from typing import Optional
+from humps import decamelize
+
+
+def to_snake(string):
+    return decamelize(string)
+
+
+class SnakeModel(BaseModel):
+    class Config:
+        alias_generator = to_snake
+        allow_population_by_field_name = True
+
+
+class EntityConfigDTO(SnakeModel):
+    id: str = Field(example='0')
+    name: str = Field(example='Kitchen')
+
+
+class NotificationConfig(SnakeModel):
+    violationThreshold: Optional[int] = Field(0, example=100)
+    notifyEveryMinutes: Optional[int] = Field(0, example=15)
+    emails: Optional[str] = Field("", example='john@email.com,doe@email.com')
+    enableSlackNotifications: Optional[bool] = Field(False, example=False)
+    dailyReport: Optional[bool] = Field(False, example=True)
+    dailyReportTime: Optional[str] = Field('06:00', example='06:00')
+
+
+class OptionalSectionConfig(SnakeModel):
+    name: str = Field(example="objects_filtering")
+    enabled: bool

api/models/camera.py

@@ -0,0 +1,47 @@
+import numpy as np
+import cv2 as cv
+
+from pydantic import BaseModel, Field, validator
+from typing import List, Optional
+
+from .base import EntityConfigDTO, NotificationConfig, SnakeModel
+
+
+class CameraDTO(EntityConfigDTO, NotificationConfig):
+    videoPath: str = Field(example='/repo/data/softbio_vid.mp4')
+    tags: Optional[str] = Field("", example='kitchen,living_room')
+    image: Optional[str] = Field("", example='Base64 image')
+    distMethod: Optional[str] = Field("", example='CenterPointsDistance')
+
+    @validator('videoPath')
+    def video_must_be_valid(cls, video_uri):
+        error = False
+        input_cap = cv.VideoCapture(video_uri)
+
+        if input_cap.isOpened():
+            _, cv_image = input_cap.read()
+            if np.shape(cv_image) == ():
+                error = True
+        else:
+            error = True
+
+        input_cap.release()
+        if error:
+            raise ValueError('Failed to load video. The video URI is not valid')
+        else:
+            return video_uri
+
+
+class CamerasListDTO(SnakeModel):
+    cameras: List[CameraDTO]
+
+
+class ImageModel(BaseModel):
+    image: str
+
+    class Config:
+        schema_extra = {
+            "example": {
+                "image": "data:image/jpg;base64,iVBORw0KG..."
+            }
+        }

api/models/classifier.py

@@ -0,0 +1,12 @@
+from pydantic import Field
+from typing import Optional
+
+from .base import SnakeModel
+
+
+class ClassifierDTO(SnakeModel):
+    device: str = Field(example="EdgeTPU")
+    name: str = Field(example="OFMClassifier")
+    imageSize: str = Field(example="45,45,3")
+    modelPath: Optional[str] = Field(example="/repo/data/custom-model")
+    minScore: float = Field(0.75, example=0.5)

api/models/config.py

@@ -0,0 +1,65 @@
+from pydantic import BaseModel, Field
+from typing import List, Optional
+
+from constants import PROCESSOR_VERSION
+
+from .app import AppDTO
+from .api import ApiDTO
+from .area import AreaConfigDTO
+from .area_logger import AreaLoggerDTO
+from .base import SnakeModel
+from .camera import CameraDTO
+from .classifier import ClassifierDTO
+from .core import CoreDTO
+from .detector import DetectorDTO
+from .periodic_task import PeriodicTaskDTO
+from .source_logger import SourceLoggerDTO
+from .source_post_processor import SourcePostProcessorDTO
+from .tracker import TrackerDTO
+
+
+class ConfigDTO(SnakeModel):
+    app: AppDTO
+    api: ApiDTO
+    core: CoreDTO
+    cameras: List[CameraDTO]
+    areas: Optional[List[AreaConfigDTO]] = []
+    detector: DetectorDTO
+    classifier: Optional[ClassifierDTO]
+    tracker: TrackerDTO
+    sourcePostProcessors: List[SourcePostProcessorDTO]
+    sourceLoggers: List[SourceLoggerDTO]
+    areaLoggers: Optional[List[AreaLoggerDTO]] = []
+    periodicTasks: Optional[List[PeriodicTaskDTO]] = []
+
+
+class GlobalReportingEmailsInfo(BaseModel):
+    emails: Optional[str] = Field("", example='john@email.com,doe@email.com')
+    time: Optional[str] = Field("06:00")
+    daily: Optional[bool] = Field(False, example=True)
+    weekly: Optional[bool] = Field(False, example=True)
+
+    class Config:
+        schema_extra = {
+            "example": {
+                "emails": "john@email.com,doe@email.com",
+                "time": "06:00",
+                "daily": True,
+                "weekly": True
+            }
+        }
+
+
+class ConfigInfo(BaseModel):
+    version: str
+    device: str
+    has_been_configured: bool
+
+    class Config:
+        schema_extra = {
+            "example": {
+                "version": PROCESSOR_VERSION,
+                "device": "device",
+                "has_been_configured": True
+            }
+        }