|
Hi! I'm Ayaan Haque, a 20 y/o researcher. I'm currently a student researcher at Google DeepMind working on video models. Previously, I spent over a year as a research scientist at Luma AI. I worked on the training team for Dream Machine, Luma's video generation foundation model! Before Dream Machine, I worked on Genie, Luma's 3D foundation model. I'm also a student at UC Berkeley studying EECS, where I've worked with Angjoo Kanazawa. In the past, I worked on self-supervised and unsupervised representation learning. I interned at Samsung SDSA, and got my research career jumpstarted back in high school with Wang Group at Stanford.In a past life, I was a builder and hacker (I'm a MLH Top-50 Hacker!), and now I'm exploring deep-tech startups. Other than that, I enjoy writing, watching/playing sports, eating out with friends, and just having a good time. My ongoing goal and dream:
Twitter / Email / Google Scholar / Github / LinkedIn |
Learning about learning 💯
|
|
|
|
|
|
|
Dream Machine is a video generation model, built on a highly scalable and efficient transformer model trained directly on videos. Dream Machine is capable of generating physically accurate, consistent and eventful shots of 120 frames in under 120 seconds, and is a first step towards building a universal imagination engine. |
|
|
Genie is a 3D foundation model that can generate high-fidelity 3D objects from text instructions. Genie generates objects in under 10 seconds, and can be refined into higher quality assets. |
|
Since I've worked on a wide variety of topics (and am still exploring new topics), I've split my publications into currently relevant works and previous works. Only relevant works are listed below. For a full list (and chronological list) of my papers, visit my Google Scholar. |
|
|
Ayaan Haque, Matthew Tancik, Alexei A. Efros, Aleksander Holynski, Angjoo Kanazawa UC Berkeley ICCV, 2023 (Oral Presentation) Project Page / ArXiv / Oral / Code We propose a method for editing NeRF scenes with text-instructions. Given a NeRF of a scene and the collection of images used to reconstruct it, our method uses an image-conditioned diffusion model (InstructPix2Pix) to iteratively edit the input images while optimizing the underlying scene, resulting in an optimized 3D scene that respects the edit instruction. |
|
Ayaan Haque, Hankyu Moon, Heng Hao, Sima Didari, Jae Oh Woo, Patrick Bangert Samsung SDS Research America AAAI, 2023 ArXiv We introduce self-supervised MeshCNN, or SSL-MeshCNN, a novel mesh-specialized contrastive learning method to perform downstream segmentation with limited-labeled data. We create an augmentation policy tailored for meshes, enabling the network to learn efficient visual representations through contrastive pre-training. |
|
Ayaan Haque Saratoga High School AAAI, 2021 (Best Student Abstract Finalist, Oral Presentation) Project Page / ArXiv / Oral / Code / Blog We propose EC-GAN, which combines a Generative Adversarial Network with a classifier to leverage artifical GAN generations to increase the size of restricted, fully-supervised datasets using semi-supervised algorithms. Mentored by Microsoft Postdoc and Princeton University PhD Jordan T. Ash. |
|
|
|
Ayaan Haque*1, Viraaj Reddi*1, Tyler Giallanza2 Saratoga High School1, Princeton University2 ICANN, 2021 Project Page / ArXiv / Teaser Video / Code / Blog We propose SDCNL to address the unexplored problem of classifying between depression and more severe suicidal tendencies using web-scraped data. Our method introduces a novel label correction method to remove inherent noise in web-scraped data using unsupervised learning combined with a deep-learning classifier based on pre-trained transformers. |
|
Ayaan Haque*1, Abdullah-Al-Zubaer Imran*2,3, Adam Wang2, Demetri Terzopoulos3,4 Saratoga High School1, Stanford University2, University of California, Los Angeles3, VoxelCloud Inc.4 IEEE ISBI, 2021 Project Page / ArXiv / Code / Blog We introduce MultiMix, a joint semi-supervised classification and segmentation model employing a confidence-based augmentation strategy for semi-supervised classification along with a novel saliency bridge module that guides segmentation and provides explainability for the joint tasks. |
|
Ayaan Haque1, 2, Adam Wang2, Abdullah-Al-Zubaer Imran2 Saratoga High School1, Stanford University2 SPIE Medical Imaging (SPIE), 2022 Project Page / Paper / Code We propose Noise2Quality (N2Q), a novel, self-supervised IQA model which predicts SSIM Image Quality maps from low-dose CT. We propose a self-supervised regularization task of dose-level estimation creating a multi-tasking framework to improve performance. |
|
Ayaan Haque1, 2, Adam Wang2, Abdullah-Al-Zubaer Imran2 Saratoga High School1, Stanford University2 MICCAI MLMI, 2021 Project Page / ArXiv / Code / Blog We introduce SSWL-IDN, a novel self-supervised CT denoising window-level prediction surrogate task. Our method is task-relevant and related to the downstream task, yielding improved performance over recent methods. |
|
Ayaan Haque1, 2, Abdullah-Al-Zubaer Imran2,3, Adam Wang2, Demetri Terzopoulos3,4 Saratoga High School1, Stanford University2, University of California, Los Angeles3, VoxelCloud Inc.4 MELBA, 2021 Project Page / Journal Page / Paper / Code We expand upon MultiMix (in ISBI 2021). Our extended manuscript contains a detailed explanation of the methods, saliency map visualizations from multiple datasets, and quantitative (performance metrics tables) and qualitative (mask predictions, Bland Altman plots, ROC curves, consistency plots). |
|
Some activities I used to really enjoy. |
|
Team Captain May 2019 - June 2022
|
|
I've just listed a few of my favorite projects, and the remaining are available on my Github. Check out this cool commit graph. |
|
|
Instruction-based editing of Gaussian Splatting scenes Nov 2023 - Jan 2024 Website / Paper / Github We propose a method for editing 3D Gaussian Splatting (3DGS) scenes with text-instructions. This project is a follow-up to our previous work, Instruct-NeRF2NeRF. We improve the visual quality of edits, the training time, and rendering speed of the model. Stack: Python, PyTorch |
|
|
A collaboration friendly studio for NeRFs Jan 2023 - May 2023 Website / Github Implemented DreamFusion, CLIP-NeRF in Nerfstudio, an open-source project for NeRF development. Contributed (in small parts) to large-scale open-source project by reviewing code, writing documentation, and implementing research methods. Stack: Python, PyTorch |
|
Streamlined prescription analysis for visually impaired patients (Available on the App Store) September 2019 - June 2020 Website / App Store / Demo / Github / Devpost Tickbird is an advanced Swift mobile app allowing visually impaired patients to aurally understand their prescriptions or the labels on their pill bottles. Stack: Swift, Xcode, IOS, Firebase, TesseractOCR, Ruby |
|
I used to write a lot on Medium, here are some of my favorite stories: |
|
October 6th, 2021 A response to an article discussing our purpose in this world combined with a discussion of my own purpose This article is a reponse to my friend's article, where he discusses critiques of our Saratoga society. In my article, I respond to his ideas and then share my own story of finding my purpose in life. |
|
Better Programming August 28th, 2020 Coding, winning prizes, and proving ourselves Authored by Ayaan Haque, Adithya Peruvemba, Viraaj Reddi, Sajiv Shah, and Ishaan Bhandari This article travels through the journey of my team, Haleakala Hacksquad, and how we became great hackers. |
|
Website template from Jon Barron |