Work

Node Classification with Integrated Reject Option [Paper] [Code]
Uday Bhaskar, Jayadratha Gayen, Charu Sharma, and Naresh Manwani.
Published in Transactions on Machine Learning Research (TMLR) 2025.
Oral Presentation at Deployable AI Workshop, AAAI 2025.

Efficient Training Strategies for Local Differentially Private Federated Networks [arXiv] [Code]
Uday Bhaskar, Varul Srivastava, Avyukta Manjunatha Vummintala, Naresh Manwani, Sujit Gujar, et al.
Poster Presentation at Privacy Enhancing Technologies Symposium 2024.
arXiv preprint. Under Review.

MatFormer and its Scaling Laws — [Code]
- Implemented a Matryoshka Transformer from NanoGPT with nested Feed Forward Network. Trained a randomly sampled substructure of the FFN for each optimizer step.
- Fit a Scaling Law for the model validaton loss on OpenWebText Dataset. Scaling matches GPT Architcture but inference can be elastic due its structure and training procedure.

NotebookMLX — [Code]
- Implemented the podcast generation feature from NotebookLM to run locally on M-series Mac with 16GB> unified memory using MLX Framework
- Extracted and processed text from PDFs using feather language models. Generated podcast script as conversation between two hosts using Llama-8b and TTS Model Dia-1.6B to generate audio

Physics of Language Models — [Code]
- I really enjoyed the ICML 2024 Tutorial on Physics of Language Models by Zeyuan Allen-Zhu and wanted to further explore fundemental capabilities of Language Models in Knowledge Retrieval and Manipulation.
- Generated synthetic datasets bioR and bioS as mentioned in this paper. Forked NanoGPT by Karpathy Sensei and extended it for instruction tuning.
- Compared performance of these models for various training strategies like Mixed Training, Augmentation, etc. as worked in this line of work.