micro-learn: A Python Library for Embedded Machine Learning

micro-learn is a Python library for converting trained machine learning models into inference code that can run on microcontrollers in real time.

ML algorithms typically require heavy computing and memory resources in the training phase, far greater than what a typical microcontroller can offer. However, post training, many of these algorithms boil down to simple parameters that require simple arithmetic & logical operations for inference. These can easily run on microcontrollers in real time. The purpose of this library is to convert ML models (trained using scikit-learn) directly into Arduino inference code.

All the inference algorithms are optimized for microcontrollers and require least possible arithmetic computations. Division operations have been converted to multiplications since the latter is much more computationally efficient. Note that all the algorithms are exact and not approximate. Please refer to my ACM paper for theoretical and practical foundations regarding this procedure.

Lightning Talk: "Edge Cloud-Native Cluster", OPNFV Plugfest, Paris 2019

One of the biggest perks of interning in the Linux Foundation Networking (LFN) program is that you get a fully sponsored trip to an LF event. Not only that, you also get to present your work as a 10-15 minute lightning talk to a room full of industry experts! My internship concluded successfully in December 2018 and I began packing my bags for this trip in the second week of January itself.

This time around (2019), the OPNFV plugfest was not a standalone LF event. Rather, a combined OPNFV Plugfest and ONAP DDF took place for 4 whole days at Nokia Paris-Saclay. I got to attend tons of exciting talks and tutorials that were undertaken by the developers from various companies like Redhat, China Mobile, Huawei, Intel. The intern talks were scheduled early morning on the last day of the event. I've attached my presentation below which is pretty self-explanatory.

Lightning Talk: "Edge of Tomorrow: Deploying Collaborative Machine Intelligence to the Edge", Open Networking Summit, Amsterdam 2018

I was only halfway through my LFN internship when one day my mentor, Wenjing Chu, suggested that we should submit a talk proposal for the Europe edition of the Open Networking Summit (2018). He was convinced that our work on edge-cloud and collaborative machine learning was something that companies would be interested in. And sure enough, our proposal was accepted by LF to be presented as a 10 minute lightning talk. I've attached our presentation below which is pretty much self-explanatory.

P.s. This was the first time I'd ever given a talk in a foreign country to a room full of industry experts. A great experience indeed!

Raspberry Pi Kubernetes Cluster

As the heading suggests, this project revolves around the creation of a kubernetes cluster using 2 or more Raspberry Pi devices (ARM-based). After manually playing with docker and kubernetes on the RPi for weeks, I developed a versatile Ansible script that can create and disassemble clusters with ease. You can try it out yourself if you own at least 2 Raspberry Pi devices. An exemplar containerized app is also included in the repo which can help you get started with your own edge cluster-based projects.

A fully detailed guide and code can be found in the below link.

Hand Segmentation Using RefineNet

The aim of this project was to develop an egocentric hand segmentation model using RefineNet in Python3. For this, we worked with two CVPR papers and replicated the results using our own implementation in PyTorch (the authors had written the code in Matlab). The PyTorch implementation of RefineNet (which is a multipath refinement network over ResNet) was done from scratch. After that, the model was trained on the standard Pascal person-parts dataset with a new binary layer of hand and no-hand. The model was further fine-tuned on four different ego-centric hands datasets.

The details about the implementation and the code can be found below.

Paper Presentation: "Machine Learning-Based Occupancy Estimation Using Multivariate Sensor Nodes", IEEE Globecom, Abu Dhabi 2018

This was my first international paper. As a rule, if an IEEE or ACM paper gets accepted for publication, at least one of the authors is expected to attend the conference and present the paper.

Abstract: In buildings, a large chunk of energy is spent on heating, ventilation and air conditioning systems. One way to optimize their usage is to make them demand-driven depending on human occupancy. This paper focuses on accurately estimating the number of occupants in a room by leveraging multiple heterogeneous sensor nodes and machine learning models. For this purpose, low-cost and non-intrusive sensors such as CO2, temperature, illumination, sound and motion were used. The sensor nodes were deployed in a room in a star configuration and measurements were recorded for a period of four days. A regression based method is proposed for calculating the slope of CO2, a new feature derived from real-time CO2 values. Supervised learning algorithms such as LDA, QDA, SVM and random forest (RF) were used on several different combinations of feature sets. Moreover, multiple performance metrics such as accuracy, F1 score and confusion matrix were used to evaluate the performance of our models. Experimental results demonstrate a maximum accuracy of 98.4% and a high F1 score of 0.953 for estimating the number of occupants in the room. PCA was also applied to evaluate the performance of a dataset with reduced dimensionality.

The presentation and the paper link can be found below.

World Rank 24 at NASA's CanSat Competition 2018

Worked part-time for around 5 months on the CanSat 2018 mission statement along with 8 other IIITians.

The mission statement of CanSat 2018 was to construct a small satellite containing various sensors such as temperature, pressure, accelerometer, voltage and GPS, a microcontroller and a communication radio. A payload section was also to be designed for protecting a delicate instrument. During the final launch, an egg was put in this section. The CanSat was also armed with a heat shield and a parachute. As per the problem statement, the CanSat was to be dropped from a height of around 700m. It was expected to perform a series of manoeuvres such as letting go of the heat shield and opening the parachute at certain altitudes, sending telemetry data to the ground control every second and most importantly, remain steady enough to ensure that the egg in the payload section doesn't break even after impact.

All of us in the CanSat team grew up watching science fiction movies. To create something like this from scratch was an amazing experience. I still can't explain the feeling I got when I learnt that the egg in our payload section didn't even have a scratch on it! For more details, please visit the following link.

Song Popularity Predictor

The aim of this project was to determine the popularity of songs using its musical features as well as other song metrics. We started with the analysis of the million song dataset (MSD). The dataset required a lot of preprocessing before it could be used in any way. After that, traditional machine learning models were trained to classify songs into three categories- highly popular, popular and unpopular. Nearest neighbor exhibited the highest accuracy. Next, we moved onto time-series based experiments as the songs are, of course, time-series in nature. An LSTM was trained on the time-series features that we had earlier left out. Since the MSD dataset was quite old, we collected our own dataset consisting of 889 current hit songs and tested different models on that too. The features were extracted using the PyAudioAnalysis library.

The details about the implementation and the code can be found below.

Faster Matrix Algebra Library For Symmetric Matrices In C++ Eigen

This project was given as a GSoC project by the ATLAS team (part of the Large Hadron Collider project) in 2018. I was initally trying for this internship but then ended up taking the Linux Foundation internship instead. However, I did complete most of the tasks laid out for the project on my own.

In this project, a standalone C++ class was built which can work with Symmetric Matrices on top of the Eigen Library. Working only with the upper/lower triangular part of a symmetric matrix considerably reduces the storage space as well as the complexity for different matrix operations. The code exhibits the use of templates coupled with operator overloading.