About Me

I am a doctoral candidate in the Computer Engineering department at George Washington University, under the guidance of Professor Guru Venkataramani. My primary research interests are software security and efficient machine learning in systems. My current research is focused on developing hardware/software optimizations for accelerating multi-agent reinforcement learning workloads. Prior to enrolling at GWU, I completed my undergraduate studies in Computer Science and Engineering at Vellore Institute of Technology, Tamil Nadu, India in 2019.

Education

• Ph.D. in Computer Engineering, The George Washington University, DC, USA.
  • Expected graduation: late 2024/early 2025
• B.Tech. in Computer Science & Engineering, Vellore Institute of Technology, 2019

Skills

• Languages: Python, R, C/C++, Bash scripting
• Tools: Weights & Biases, PowerBI, Tableu, RAPL, Vim, LaTex, Visual Studio, Perf, Anaconda, Jupyter Notebook, Google colab, Numba, Microsoft Excel, Google Analytics
• Databases: MySQL, Microsoft SQL
• Cloud Technologies: Google Cloud Platform, Amazon Web Services
• Frameworks: Scikit-learn, Tensorflow/Keras, Pytorch, NLTK, Pandas, Numpy, Matplotlib
• Statistics/ML: Linear & Logistic regression, Random Forest, SVM, Neural Networks, CNN, XGBoost, Naive Bayes Classifier
• Platforms: Windows, Linux

Research Projects

• High-Performance Multi-Agent Reinforcement Learning (MARL)

  MARL is a promising research area that can model and control multiple distributed decision-making AI agents. However, recent studies have shown that the MARL algorithms suffer from inefficiencies that can severely limit their adoption in real-world systems. These problems occur due to complexities in decision-making processes arising from having to observe and act upon a large number of events present in the environment, along with the growth in the number of AI agents needed to interact with each other. To ameliorate the learning efficiency and scalability issues of MARL algorithms, we will seek techniques to improve neural-network throughput, to efficiently manage the state-action space in a dynamic fashion and to scalably encode states and observations of a large and varying number of agents.

• Hardware-assisted Self Tuning Cyber Deception

  The objective of this project is to design a secure micro-architecture ecosystem that effectively deceives adversaries in an automated fashion and drains their resources ultimately in order to prevent bad actors from gaining access to sensitive information. The investigation will span memory, computer, and other performance enhancing hardware (including the various predictors). This project will add a unique dimension to computer system security by complementing the trusted execution environments with active defense via hardware.

• Protocol Customization via Feature diagnosis, Elimination, Cross-grafting, and Trimming

  The project will investigate how to individualize security in cyber systems by customizing their protocols. In contrast to the currently adopted “protocol standards", this first-of-a-kind approach is motivated by the fact that unnecessary code/layer often introduced by standards may eventually be used as backdoors for security exploits, while protocol customization enables the feature to debloat and can significantly eliminate the risks associated with monoculture cyber systems.

Experience

• 2019 – Present: Graduate Research Assistant
  The George Washington University, Advisor: Dr. Guru Venkataramani.
  • High Performance Multi-Agent Reinforcement Learning (MARL)
    1. Developed AccMER (ASAP'23), that repeatedly reuses the transitions for a window of n steps in order to improve the cache locality and minimize the transition data movement, instead of sampling new transitions at each step. Our experimental results on the Predator-Prey environment demonstrate the effectiveness of reusing the essential transitions based on the priority weights, where we observe an end-to-end training time reduction of 25.4% (for 32 agents) compared to existing prioritized MER algorithms without notable degradation in the mean reward.
    2. Analyzed the computational and scalability bottlenecks of Multi-Agent algorithms from systems perspective. Furthermore, we provide suggestions on future architectural improvement such as primitives for selective attention computation, shared-memory multi-threading, and processing-in-memory. (FastPath'23)
    3. Explored a neighbor sampling strategy (MLArchSys-ISCA'23) to improve the cache locality and observe performance improvement ranging from 26.66% (3 agents) to 27.39% (12 agents) for the computationally intensive mini-batch sampling phase. Additionally, we demonstrate that improving the cache locality leads to an end-to-end training time reduction of 10.2% (for 12 agents) compared to existing multi-agent algorithms without significant degradation in the mean reward.
    • We proposed optimizations aim to alleviate performance bottlenecks related to irregular memory access patterns and cache misses. By employing techniques like cache-aware sampling and importance sampling, substantial training time acceleration is achieved. These methods provide valuable insights for improving MARL algorithms and reducing computational overhead. (Extended version of MLArchSys'23 - Under review)
    4. Developed SwiftRL (ISPASS'24), a framework leveraging Processing-In-Memory (PIM) architectures to enhance Reinforcement Learning (RL) training. It addresses RL challenges, such as memory-bound training and data transfer bottlenecks, by performing computations within memory devices. The study adapts Tabular Q-learning (also Multi-Agent version of Tabular Q-learning) and SARSA algorithms on UPMEM PIM systems, optimizing performance and demonstrating near-linear scaling of 15× with a 16× increase in PIM cores. Comparative evaluations against Intel CPU and NVIDIA GPU highlight the superior performance of PIMs in various implementations.
  • Highlights: Systems for ML, CPU-GPU, Run-time Performance Optimization, Workload Characterization, Quantization, Processing-In-Memory, Hardware/Software Prefetching, Multi-threading.
  • Hardware-assisted Self Tuning Cyber Deception
    1. Developed Foreseer (SEED'22), a framework that can help proactive defenses through forecasting the raw event series needed for early detection of malware. Our experimental results show that Foreseer is able to synthesize ransomware-related features with low errors for most dominant factors used in detection, and cuts the time to predict the malware presence by about 40%.
    2. Explored hardware support to efficiently implement cyber-deception to transparently counter the malware with a near-native execution profile, thereby making deception-related operations invisible to the attackers (GLSVLSI'23).
    3. MAYALOK (ASAP'23) introduces hardware deception primitives to transparently insert or skip malware program instructions during runtime and present a deceptive view of the system state to the attackers. We evaluate and demonstrate the deception efficacy of the MAYALOK framework on malware samples representing various attack vectors: Ransomware, Info-stealers, Buffer overflow, and Side-channels.
  • Highlights: Gem5, Hardware/Software Security,Pro-active Defense/Deception
  • Cyber Security: Protocol Customization via Feature diagnosis, Elimination, Cross-grafting, and Trimming
    1. Verify-Pro (MILCOM'22) aims to use the protocol dialects as fingerprints during communication, and dynamically select different dialects based on a unique request in the session to compel continuous authentication.
    2. Developed a reliable application-layer moving target defense model (MPD) via customized communication protocols and dynamic synchronization and management (Securecomm'21).
  • Highlights: Cyber Security, Machine Learning for Security, Communication Protocol Customization
• Summer 2018: SDE Intern
  
  • Developed a robust architecture for a speech decoding application, catering to a user base exceeding a million users daily.
  • Led data cleaning efforts to ensure high-quality data inputs for the application.
  • Spearheaded web scraping activities utilizing the Scrapy web crawler tool, gathering data from diverse websites to enrich the application’s content and functionality.

Publications

Professional Services

Conferences

• Submissions Chair for IEEE International Symposium on Workload Characterization (IIWSC), 2023, Ghent, Belgium

Journals

• Reviewer of the IEEE Micro (MICRO)
• Reviewer of Journal of Hardware and Systems Security

Student Volunteer

• NSF CISE CAREER Workshop 2024
• IEEE International Conference on Computer Design (ICCD 2023)

Awards

• TCO 2022 Innovation Competition, Winners of $5,000 Finalist prizes.
• University Fellowship - June 2020, 2021, 2022.
• Student Travel Grant - ISCA 2023
• Student Travel Grant - ISPASS 2024

Presentations

• Communication Protocol Customization and Fuzzing, Software Security Summer School (SSSS’20)
• Co-Presenter Twin-Finder at the International Workshop on Software Clones (SANER 2020)
• Presented our Research on Communication Protocol Customization to the Office Naval Research, 2020
• Presented Verify-Pro at the Military Communications Conference (MILCOM 2022)
• Presented Verify-Pro in Young Scholar Workshop at MILCOM 2022
• Presented Verify-Pro (poster) at SEAS Student Research & Development Showcase, 2022.
• Presented Verify-Pro (poster) at cybersecurity & privacy@gw day
• Presented Foreseer at the ECE Research Blitz (Audience Choice Award Winner), FALL 2022.
• Scalability and Computational Bottlenecks in Multi-Agent RL Systems at FastPath'23.
• Presented our work Towards Efficient Multi-Agent Learning Systems at MLArchSys 2023 during ISCA-50, FCRC.
• Presented SwiftRL at IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS'24)

Powered by Jekyll and Minimal Light theme.