Data Science | Machine Learning | Cloud and Software Engineer

Andrew Xavier

### Project Video, Abstract, and Repository

*   [Link to abstract (paper)](https://github.com/Qulxis/ML-Bike-Break/raw/main/Paper_ML_Bike_Break.pdf)

*   [Link to video and demo](https://www.youtube.com/watch?v=RIn2AcNfQwM&ab_channel=KennethMucyo)

*   [Link to repository](https://github.com/Qulxis/ML-Bike-Break/tree/main)

### Description:

**Abstract:**
The project introduces a Smart Bicycle Taillight (SBT) designed to enhance cyclist safety. The SBT operates in strobe mode during deceleration and remains solid during coasting or acceleration. A unique feature of this project is the integration of a Kalman filter with a feature extractor and multi-layer perceptron (MLP) for real-time inference on low-power edge devices. The model outperforms other methods, achieving a classification accuracy of 95.6%.

![](/images/demo-f201bb0c.jpg)

**Introduction:**
Cycling safety is paramount, with over 150 preventable cycling deaths reported monthly. While taillights are a common safety feature, the integration of Deep Learning (DL) in cycling safety remains largely unexplored. This project introduces an SBT that enhances visibility, especially in professional cycling races.

**System Design:**

*   **Pipeline:**

    *   Utilizes Arduino Nano 33 BLE with a built-in accelerometer.

    *   Sensor values are processed through a Kalman filter to reduce noise.

    *   A sliding window captures the last 120 samples for prediction.



*   **Model Design:**

    *   The system integrates a Kalman-filtered accelerometer data into a 1-D convolutional feature extractor.

    *   The model is lightweight, ensuring low computational cost and high performance.

![](/images/model%20architecture.png)

**Data Acquisition:**

*   **Data Collection:**

    *   Data was manually collected using a photo-resistor circuit.

    *   The system deployed on a bicycle differentiates between coasting/accelerating and braking based on the photo-resistor's resistance.

*   **Data Cleaning:**

    *   Erroneous labels were adjusted, and valid sequences were detected based on sequence length.

    *   Synchronization between Arduino's recordings and video recordings ensured accurate data.

*   **Data Post Processing:**

    *   Gravitational pull adjustments were made to the accelerometer data.

    *   Kalman filters were used to filter out environmental noise.

**Results:**
The model was evaluated in a controlled setting, demonstrating high accuracy in brake episode classification. Our model significantly outperforms other methods, beating the best-thresholded method by **26.5%** on the same dataset,
and achieving a test accuracy of **95.6%**

**Conclusion:**
The developed SBT has shown promising results in detecting brakes made by cyclists. With further testing and data collection, the SBT has the potential to be introduced to the consumer market, enhancing safety for all cyclists

Smart Bicycle Taillight

Multi-user playlist recommendation system

### Introduction

Music streaming platforms have revolutionized the way we consume music. Among these platforms, Spotify stands out with its 433 million active users in 2022 and a vast database of over 80 million songs. Recognizing the potential of this data-rich environment, we embarked on a project to create a custom playlist generator. Unlike Spotify's existing "Blend" feature, which only merges songs from two users' existing libraries, our tool introduces users to entirely new tracks based on their combined musical tastes.

### The Problem

Spotify's "Blend" feature allows two users to generate a shared playlist from their existing song collections. However, it doesn't introduce them to new songs that neither has heard before. Our goal was to bridge this gap and create a tool that generates playlists with novel songs based on the combined preferences of multiple users.

### Our Solution

Utilizing Spotify's developer platform and the Python library, Spotipy, we built a model that takes inputs from multiple users in the form of existing playlists. The model then generates a new playlist composed of songs that align with the users' tastes but aren't already in their libraries.

<div style="display: flex; justify-content: center;">
    <img src="/images/architecture%20(1)-e46752e1.png" width="400px">
</div>

### Data Collection and Preprocessing

Our primary dataset was sourced from a Kaggle dataset containing 26 million songs. Due to API limitations, we narrowed this down to a subset of the top 100,000 tracks. This dataset was then processed using Google Cloud Platform and Spark to extract essential features like danceability, tempo, and energy.

<div style="display: flex; justify-content: center;">
    <img src="/images/SparkCollection.png" width="400px">
</div>

### Exploratory Data Analysis (EDA)

Exploratory Data Analysis (EDA) is a foundational step in the data analysis process. It involves visualizing, summarizing, and interpreting the information that is hidden in rows and columns of data. For our Spotify playlist generator project, EDA was crucial in understanding the underlying structure of the data, the relationships between variables, and the potential challenges in modeling.

#### 1. Data Distribution:

*   **Genre Distribution:** One of the first visualizations we created was a histogram of song genres. This helped us understand the most prevalent genres in our dataset. As observed, pop songs dominated the dataset, which is reflective of global music trends.

<div style="display: flex; justify-content: center;">
    <img src="/images/Data1.png" width="400px">
</div>

#### 2. Feature Relationships:

*   **Feature Correlation:** We examined the correlation between different song features. For instance, there was a positive correlation between energy and loudness, indicating that louder songs tend to be more energetic. Conversely, energy and acousticness were negatively correlated.

<div style="display: flex; justify-content: center;">
    <img src="/images/Data2.png" width="400px">
</div>

*   **Energy vs. Loudness:** A level curve graph was plotted to delve deeper into the relationship between energy and loudness. This visualization reaffirmed our earlier observation: songs with higher loudness levels also tend to have higher energy.

<div style="display: flex; justify-content: center;">
    <img src="/images/Data3.png" width="400px">
</div>

#### 3. Song Characteristics:

*   **Polar Graph of Audio Features:** To get a sense of what might make a song popular or resonate with listeners, we plotted a polar graph of various audio features. Danceability emerged as a significant factor, suggesting that users might prefer songs they can dance to.

<div style="display: flex; justify-content: center;">
    <img src="/images/Data4.png" width="400px">
</div>

*   **Danceability vs. Tempo:** We visualized songs based on their danceability and tempo, color-coded by genre. This visualization revealed genre-specific trends, such as trap Latino songs having higher danceability compared to rap songs.

#### 4. Clustering Attempts:

*   **K-means Clustering:** We attempted to cluster songs using K-means to see if distinct clusters would emerge based on genres or other features. However, the results were not as distinct as we had hoped, indicating the complexity and multi-dimensionality of our dataset.

<div style="display: flex; justify-content: center;">
    <img src="/images/Data6.png" width="400px">
</div>

#### 5. Data Sparsity:

**Dataset Overview:** A significant observation was the sparsity of our dataset. This was crucial as it informed our decision to not rely heavily on density-based clustering methods, which might not perform well on sparse datasets.

<div style="display: flex; justify-content: center;">
    <img src="/images/Sparse.png" width="400px">
</div>

### The Models

#### Baseline Model:

Our initial approach used variance-based feature selection. By analyzing the variance within the features of tracks in a user's playlist, we identified the most consistent qualities in their music taste. Using these features, we then selected songs from our library that closely matched this profile.

#### K-Nearest Neighbors (KNN):

We also experimented with KNN, using grid search to optimize the number of neighbors. This model looked at the closest songs in feature space to the user's preferences to generate the playlist.

#### Random Forest Classifier:

Our final and most effective model was the Random Forest Classifier. This model classified songs as "good" or "bad" based on the combined preferences of the users. It then generated a playlist of the top "good" songs.

### Web Application

To make our tool accessible, we developed a web interface where users can log in with their Spotify accounts, select their preferred playlists, and receive a custom-generated playlist. This playlist can then be saved directly to their Spotify account or played on our platform.

<div style="display: flex; justify-content: center;">
    <img src="/images/Inteface2.png" width="400px">
</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/Interface3%20(1)-3693dd28.png" width="500px">
</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/Interface6.png" width="500px">
</div>

### Conclusion

Our Collaborative Spotify Playlist Generator offers a unique solution to music lovers looking to discover new tracks that align with their tastes. By leveraging the power of big data and machine learning, we've created a tool that enhances the music streaming experience, making it more social and exploratory.

Please refer to the completed Github repository: <https://github.com/Qulxis/spotify-big-data-project>


Collaborative Spotify Playlist Generator 

Real-Time ASL detection, correction, & completion on resource constrained devices

*By Andrew Xavier, Alban Dietrich, Kenneth Munyuza, and Sam D. Brosh, Columbia University*

### Video, Paper, and Repository

*   [Project Video Link](https://www.youtube.com/watch?v=4rL-yoksNag&ab_channel=KennethMucyo)

*   [Paper Link](https://github.com/Kennethm-spec/ASL-Auto-Complete/blob/main/ADL_Final_Project.pdf)

*   [Repository Link](https://github.com/Kennethm-spec/ASL-Auto-Complete)

**Introduction**

American Sign Language (ASL) serves as a vital communication bridge for the deaf community. While ASL primarily uses hand gestures instead of spoken words, its detection has become a focal point in image processing and classification. Recognizing the challenges and the tedious learning curve associated with ASL, our team introduced the ASL Detection, Correction, and Completion (ASLDCC) application. This innovative tool, compatible with both computers and smartphones, aims to make ASL communication more accessible and efficient.

### **About ASLDCC**

ASLDCC is equipped with three main submodules:

1.  **Detection Submodule**: Recognizes the user's hand signs.

2.  **Natural Language Processing (NLP) Submodule**: Suggests corrections or completions based on detected signs.

3.  **Full-Stack Submodule**: Presents the detected letters and recommendations in a user-friendly interface.

The application is especially beneficial for users with basic interfaces, eliminating the need for a keyboard. In an era dominated by Augmented Reality (AR), ASLDCC ensures that signers can communicate without fretting over errors or prolonged signing times.

### **How It Works**

*   **Detection**: Utilizing MediaPipe, the application detects hand landmarks in an image (in this case, frames from the live video feed). This data was used to train a convolutional neural network (CNN) to identify ASL representations of alphabet letters. During inference, the predictions are aggregated over frames to ensure the signed word is intentional

<div style="text-align: center">Figure 1: Computer Vision Detection submodule pipeline</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/detectino_module_updated-fd44cdc9.png" alt="Detection Module" width="600px">
</div>

*   **NLP Processing**: Depending on the detected signs, the NLP algorithm suggests possible word completions. For instance, if the letter "A" is detected, the algorithm might suggest "Able", "Apple", or "Apply".] The possible words for a given input are generated by using a DAWG. The top 3 words of those are determined by a statistical model using the brown corpus as a reference set

<div style="text-align: center">Figure 2: NLP submodule pipeline.</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/nlp_workflow%20(1).png" alt="NLP Processing" width="600px">
</div>

<div style="text-align: center">Figure 3: DAWG graph example using only our top 20 words.
Actual DAWG during inference uses 23,000 words</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/DAWG_graph_top20-90e8cd40.png" alt="DAWG Graph" width="600px">
</div>

*   **Full-Stack Integration**: The entire process is seamlessly integrated into a user-friendly interface, allowing users to view detected letters and select word suggestions

<div style="text-align: center">Figure 4: Demo using the Desktop Version</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/sc1_app%20(1)-7f229973.png" alt="Full-Stack Integration" width="350px">
</div>

### **Performance & Results**

Our ASL classification model, trained on diverse datasets, achieved **98.3% accuracy and an average f1-score of 0.985
across all classes**. The application's efficacy was further demonstrated through its real-time performance on both mobile and web platforms. Notably, the web application provided slightly enhanced accuracy due to the stability of laptop configurations.

<div style="text-align: center">Figure 5 + 6: Performance of CV Detection Module
</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/Confusion_matrix_testset-3b309b06.png" alt="Confusion Matrix" width="500px">
</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/Class%20scores-133b7e2b.png" alt="Class Scores" width="400px">
</div>

### Example of Final Product on Mobile Device:

<div style="text-align: center">1.  User signs "E" and intends to type "each"</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/phoneSS1-57f0895e.png" alt="Mobile Example 1" width="400px">
</div>

<div style="text-align: center">2.  User signs "2" to select the second auto-complete option, "each"</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/phoneSS2-e2c66c3f.png" alt="Mobile Example 2" width="400px">
</div>

<div style="text-align: center">3.  "Each" is added to the current result message</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/phoneSS3-835ab8c4.png" alt="Mobile Example 3" width="400px">
</div>

**Conclusion & Future Prospects**

ASLDCC stands as a stride in addressing the communication challenges faced by the deaf community.

Future enhancements include refining the dataset, introducing hand filter mechanisms to eliminate background distractions, and expanding the model to recognize full ASL words, which involve dynamic hand movements.


ASL Typing AI Platform

Using deep-learning to diagnose Covid-19 cases from patient audio

<div style="display: flex; justify-content: center;">
    <img src="/images/Example%20of%20Mel%20Spectro%20Covid%20posititve.png" alt="Mobile Example 1" width="500px">
</div>

### Paper and  Repository:

**Please refer to the paper for in depth information about this project**

*   [Link to Paper](https://github.com/aem713/COVIDNet/raw/main/Reports/DSCI%20400%20-%20COVIDNet%20-%20Final%20Report.pdf)

*   [Link to Repository](https://github.com/aem713/COVIDNet)

### Description

At the start of 2021, the covid-19 pandemic was in full swing. In response, our junior data science team endeavored to diagnose covid-19 from patient cough audio using deep learning. We used a  labeled set of 10,000+ recordings of patients coughing and breathing. Our fundamental approach was to transform the audio recordings in Mel-Spectrograms: 2-dimensional frequency graphs in the Mel scale as shown above. We then used those as inputs to train a convolutional neural network to run in parallel with a Single-Layer Perceptron that used handcrafted features as inputs.

<div style="text-align: center">Figure 1: CNN Structure Used</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/cnn_diagram_horiz3.png" alt="Mobile Example 1" width="500px">
</div>

<div style="text-align: left"></div>

<div style="text-align: center">Figure 2: Aggregate Model</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/ending_diagram.png" alt="Mobile Example 1" width="400px">
</div>

Our project went through the full data science pipeline. By wrangling the data, we converted the data into usable training, testing, and validation sets.

<div style="text-align: center">Figure 3: Reorganization of Data</div>

<div style="display: flex; justify-content: center;">
    <img src="/images/fileorg.png" alt="Mobile Example 1">
</div>
Through data exploration, we carefully analyzed the demographic and symptom association of the patients to better understand patterns and better understand feature importance. By looking at previous work, we found promise in using ImageNet for transfer learning to accommodate for our dataset's relatively small size. We documented our work using [Github](https\://github.com/aem713/COVIDNet) and wrote a final research paper on our project. This project proved to be a valuable experience personally as we had guidance from Dr. Arko Barman to ensure proper procedure and direction.

COVIDNet was my first start-to-finish data science project.


CovidNet: MEl Spectrograms and Machine LEarning

Visualize patterns from select episodes, highlighting the show's core themes and emotions

## What is On "Off the Air" ?

On "Off the Air" is our tribute to Dave Hughes and the hundreds of artists involved in creating the various compositions incorporated in each episode of Off the Air. We have created four sets of data visualizations using audio and video from the show to allow for various ways to find patterns, themes, and correlations latent within and between episodes of the series.

For viewers of the show, we hope this project will provide you unique modalities to explore the themes and ideas present in Off the Air helping you connect the ideas of various episodes together in new ways. For those new to this show, we hope this project will allow you to explore the meaning of these topics --- and how they relate to each other -- more generally, while also motivating you to try out the show.

To make data processing and comparison as comprehensive as possible, this project focuses on the following 9 episodes: Bliss, Conflict, Coping, Death, Dreams, Family, Love, Nightmares, and Paradise. We chose these episodes since, to us, these topics best reflect the central emotions and experiences to humans and their interactions with each other.

## Project Contents

*   [Explore the Rainbow](https://observablehq.com/@amir-lankarani/on-off-the-air_explore-the-rainbow) - Compare how different colors are used in each episode

<div style="display: flex; justify-content: center;">
    <img src="/images/off%20the%20air%20section%201%20ss.jpg" alt="Explore the Rainbow" width="400px">
</div>

*   [Visualize the Music](https://observablehq.com/@amir-lankarani/on-off-the-air_visualize-the-audio) - See real-time audio data as it interacts with image frames

<div style="display: flex; justify-content: center;">
    <img src="/images/On%20Off%20the%20Air%20SS%20Audio.jpg" alt="Visualize the Music" width="400px">
</div>

*   [Connect the Frames](https://observablehq.com/@amir-lankarani/on-off-the-air_connect-the-frames) - Explore how similar frames are between episodes and get time-stamped episode recommendations based on a desired set of colors

<div style="display: flex; justify-content: center;">
    <img src="/images/connecting%20the%20frames.jpg" alt="Connect the Frames" width="400px">
</div>

*   [Connect the Episodes](https://observablehq.com/@amir-lankarani/on-off-the-air_connect-the-episodes) - View each episodes' data values alongside each other

<div style="display: flex; justify-content: center;">
    <img src="/images/Screenshot%202023-08-24%20221809.png" alt="Connect the Episodes" width="400px">
</div>

[Link to view full Project](https://observablehq.com/@amir-lankarani/on-off-the-air)


Multimodal Data Visualization: "On "Off the Air"

Predicting Solar Panel Power Generation using Machine Learning and Big Data

Article In Progress

Please refer to the completed Github Repo: <https://github.com/Qulxis/ML-Climate-Project-ahx2001>

![Placeholder Image](https://assets.stackbit.com/components/images/default/post-4.jpeg)


Solar Power Generation Modeling

Recent Projects

Machine Learning and Data Science

"Sleek, hands-free electric longboard that accelerates, brakes as riders shift weight"

###### Creating an aesthetic, fully functional, pressure controlled, electric longboard was a highlight of my senior year.

###### It winning the top prize out of all 60+ teams in that year's *Engineering in George R. Brown Engineering Design Showcase* was the icing on the cake.

[Click here to watch our board in action!](https://www.youtube.com/watch?v=pEpF\_1QBUQ0\&ab_channel=RiceUniversity)

[Click here for the official news article on our project](https://news.rice.edu/news/2022/rice-u-students-develop-sleek-hands-free-electric-longboard-accelerates-brakes-riders)



Breadboard

Projects

Engineering

Stay up-to-date with my words ✍️


Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed ante lorem, tincidunt ac leo efficitur, feugiat tempor odio. Curabitur at auctor sapien. Etiam at cursus enim. Suspendisse sed augue tortor. Nunc eu magna vitae lorem pellentesque fermentum. Sed in facilisis dui. Nulla molestie risus in mi dapibus, eget porta lorem semper. Donec sed facilisis nibh. Curabitur eget dui in libero euismod commodo nec sit amet est. Etiam id ipsum aliquam, vehicula erat sit amet, consequat tortor.

Etiam facilisis lacus nec pretium lobortis. Praesent dapibus justo non efficitur efficitur. Nullam viverra justo arcu, eget egestas tortor pretium id. Sed imperdiet mattis eleifend. Vivamus suscipit et neque imperdiet venenatis. In malesuada sed urna eget vehicula. Donec fermentum tortor sit amet nisl elementum fringilla. Pellentesque dapibus suscipit faucibus. Nullam malesuada sed urna quis rutrum. Donec facilisis lorem id maximus mattis. Vestibulum quis elit magna. Vestibulum accumsan blandit consequat. Phasellus quis posuere quam.

## About Coding

> “Coding is the language of the future, and every girl should learn it. As I've learned from watching girls grow and learn in our classrooms, coding is fun, collaborative and creative.”

Vestibulum ullamcorper risus auctor eleifend consequat. Vivamus mollis in tellus ac ullamcorper. Vestibulum sit amet bibendum ipsum, vitae rutrum ex. Nullam cursus, urna et dapibus aliquam, urna leo euismod metus, eu luctus justo mi eget mauris. Proin felis leo, volutpat et purus in, lacinia luctus eros. Pellentesque lobortis massa scelerisque lorem ullamcorper, sit amet elementum nulla scelerisque. In volutpat efficitur nulla, aliquam ornare lectus ultricies ac. Mauris sagittis ornare dictum. Nulla vel felis ut purus fermentum pretium. Sed id lectus ac diam aliquet venenatis. Etiam ac auctor enim. Nunc velit mauris, viverra vel orci ut, egestas rhoncus diam. Morbi scelerisque nibh tellus, vel varius urna malesuada sed. Etiam ultricies sem consequat, posuere urna non, maximus ex. Mauris gravida diam sed augue condimentum pulvinar vel ac dui. Integer vel convallis justo.

## Productivity

Nam rutrum magna sed pellentesque lobortis. Etiam quam mauris, iaculis eget ex ac, rutrum scelerisque nisl. Cras finibus dictum ex sed tincidunt. Morbi facilisis neque porta, blandit mauris quis, pharetra odio. Aliquam dictum quam quis elit auctor, at vestibulum ex pulvinar. Quisque lobortis a lectus quis faucibus. Nulla vitae pellentesque nibh, et fringilla erat. Praesent placerat ac est at tincidunt. Praesent ultricies a ex at ultrices. Etiam sed tincidunt elit. Nulla sagittis neque neque, ultrices dignissim sapien pellentesque faucibus. Donec tempor orci sed consectetur dictum. Ut viverra ut enim ac semper. Integer lacinia sem in arcu tempor faucibus eget non urna. Praesent vel nunc eu libero aliquet interdum non vitae elit. Maecenas pharetra ipsum dolor, et iaculis elit ornare ac.

Aenean scelerisque ullamcorper est aliquet blandit. Donec ac tellus enim. Vivamus quis leo mattis, varius arcu at, convallis diam. Donec ac leo at nunc viverra molestie ac viverra nisi. Proin interdum at turpis at varius. Nunc sit amet ex suscipit, convallis ligula eu, pretium turpis. Sed ultricies neque vel mi malesuada, et mollis risus lobortis. Sed condimentum venenatis mauris, id elementum dolor gravida ac. Sed sodales tempus neque, quis iaculis arcu tincidunt ut. Donec vitae faucibus dui. In hac habitasse platea dictumst. Donec erat ex, ullamcorper a massa a, porttitor porta ligula.


Coming soon

Posts

Contact directly at: ahx2001@columbia.edu


Home