Tiny_Heat_Face_Recognition

TinyML heat image face recognition demo

Edge-Impulse project link :https://studio.edgeimpulse.com/public/37746/latest/learning/keras/9

Getting Started

Clone the repository

git clone https://github.com/CaptainKAZ/Tiny_Heat_Face_Recognition.git --recurse-submodules

Part I Preperation

Hardware: Wio Terminal, Grove - Thermal Imaging Camera (MLX90640) (Both FOV 110° and FOV 55° is acceptable) & SD Card. Software: Arduino & Edge Impluse.

Part II Theory:

Thermal Imaging:

With global outbreak of COVID-19, we are more and more fami familiar with thermal imaging as it is part of the first line defencing the virus. Thermographic, or, thermal imaging cameras usually detect radiation in the long-infrared range of the electromagnetic spectrum (roughly 9,000–14,000 nanometers or 9–14 μm) and produce images of that radiation, called thermograms. Since infrared radiation is emitted by all objects with a temperature above absolute zero according to the black body radiation law, thermography makes it possible to see one's environment with or without visible illumination. The amount of radiation emitted by an object increases with temperature; therefore, thermography allows one to see variations in temperature. Now we are not going to use the perfessional version of thermal imaging camera with high resolution and accuracy, but using a simple and inexpensive camera sensor module Grove - Thermal Imaging Camera (MLX90640). The module we use has only 32*24 (768) thermal pixels, it is hard for human eye to even recognize object through raw data, but it is optimal for tiny ML magic. With edge impluse and tiny ML we make it possible to recognize people's face.

Face Recognition:

As mentioned above, thermal camea is good at captureing the difference in tempreture, our face consist of many muscles, which generates differerent amout of heat and provide enough temperature difference. The muscles shape our face, which also means that we can use ML to classify between people. Un like visible imaging, thermal imaging can't be cheated by masks or photos, because they do not imitate the distribution of heat, which provides higher insuarance on security.

Flash Firmware

Install Seeed_Arduino_MLX9064x, Seeed_Arduino_LCD, Seeed_Arduino_FS and Seeed_Arduino_SFUD. We also need a small lib TinyJPEG to encode and save our data.

Acquiring the data

Plugin the Grove - Thermal Imaging Camera (MLX90640) and insert SD card, Flash the sketch, and if everything is OK you will see the thermal image on the screen. Press A B and C button to capture with different label. Default label is face 0 for A, face 1 for B, environment for C (can be modified on program above). Once you press the button, you will see a number of the same label samples collected and the current label on the screen, and a jpg file is saved to your SD card. Press A to start "Environment" data acquisition. Once you press the button A, you will see a number of the same label samples collected and the current label "Environment" on the screen, and a jpg file is saved to your SD card. Note that for accurate data acquisition, the reflash rate of the sensor is relatively slow. You should hold the sensor until the image is stable before start capturing. Wrong example:

Uploading the data

After collecting about 200 samples for each labels, we can turn off the power and plug the SD card to our compueter. If everything goes right, there should be jpeg files on your disk. Face image is collected in grey scale. Open Edge Impulse and create a new project. Upload your files in data acquisition tag. Select all your sample and upload to edge impulse After that you should see your data on the dash board.

Part IV Model Design & Training

Impulse I created. Use image data input, raw data DSP and Kearas NN classifier. The transfer learning EI provided is not suitable for such a small image and limited computing resource. CNN architecture: The traing parameters: And the result: 100% val accuracy is not the best result but a little overfitting. Maybe it is bacause I set the model too wide instead of deep, and model just "memorize" the input and output.

Part V Model Deploy

Import the lib from the Edge-impulse and flash the Data Collection Sketch