Guanduo Chen

Guanduo Chen

Cool

About Me

I am building LLM at Moonshot AI.

Priviously, I completed a remote research internship supervised by Prof. Binhang Yuan at HKUST. Prior to that, I was a research intern at NTU, where I worked under the mentorship of Prof. Siqiang Luo.

Industry Experiences

  • Training System Engineering Intern
    Meituan, Large Model Architecture Team, (01/2025 - 03/2025)
    • Vision Encoder Long Context Training Support: Designed and developed a context-parallelism mechanism for the vision encoder of a large multi-modality model.
  • Database System R&D Intern
    ByteDance, ByteHouse Runtime Team, (12/2023 - 04/2024)
    • Geographic Data Aggregation Query Benchmark: Developed and benchmarked geospatial aggregation queries on the NYC Taxi dataset, identifying performance disparities across major database platforms (StarRocks, ClickHouse, PostGIS, DuckDB, and ByteHouse-CE). This analysis provided actionable insights for optimizing geospatial query execution.
    • Geospatial Data Support for CNCH: Integrated geometry data types into ByteHouse-CNCH (Cloud Native ClickHouse) via the geos library, enhancing its capabilities and improving data representation efficiency.
    • Geospatial Index Design: Led the design and implementation of multi-level indexing (disk and memory cache) for geospatial data, reducing query latency by $50\%$ compared to the base ClickHouse implementation.

Research Experiences

  • CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models [preprint]
    Hong Kong University of Science and Technology, (03/2024 - 01/2025)
    Collaborated with Prof. Binhang Yuan and Prof. Kun Yuan
    • Algorithm Development: Developed CE-LoRA, a high-efficiency algorithm for parameter-efficient fine-tuning (PEFT), which significantly reduced backpropagation costs in large language model training. By leveraging structured sparsity and low-rank approximation techniques, the model achieved a $3.39\times$ improvement in training efficiency without sacrificing accuracy.
    • Theoretical Analysis: Conducted a rigorous convergence analysis, proving that CE-LoRA maintains the same convergence rate as LoRA, but with reduced computational overhead.
  • Oasis: An Optimal Disjoint Segmented Learned Range Filter [VLDB 24]
    Nanyang Technological University, (02/2023 - 11/2023)
    Collaborated with Prof. Siqiang Luo and Dr. Meng Li
    • Oasis: Developed Oasis, a learned range filter that segments the key space into non-overlapping intervals and maps data into a bitmap using a linear model-simulated CDF as the hash function. The filter utilizes block-based Elias-Fano compression to reduce space overhead without compromising query efficiency.
    • Oasis+: Created Oasis+, a hybrid range filter that combines learning-based and hash-based methods to enhance filter applicability and robustness across various workloads.
    • Integration into RocksDB: Integrated Oasis and Oasis+ into RocksDB and tested their performance, achieving up to $6.2\times$ improvement in query response times.
  • Gar: A Generate-and-Rank Approach for Natural Language to SQL Translation [ICDE 23]
    Fudan University, (07/2021 - 02/2022)
    Collaborated with Prof. X. Sean Wang
    • Text2SQL Framework Development: Developed the GAR framework for Text2SQL translation, using a unique “Generate-and-Rank” approach that leverages parsing, generation, and ranking strategies for high-accuracy SQL generation from natural language queries.
    • Benchmarking: Built and tested a complex benchmark with self-joins, analyzing GAR’s performance against other end-to-end models, providing crucial insights into its strengths in complex query generation.

Publications

  • [preprint] CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models
    Guanduo Chen, Yutong He, Yipeng Hu, Kun Yuan, Binhang Yuan
    [Paper]

  • [VLDB 24] Oasis: An Optimal Disjoint Segmented Learned Range Filter
    Guanduo Chen, Zhenying He, Meng Li, Siqiang Luo
    [Paper, Code]

  • [ICDE 23] Gar: A Generate-and-Rank Approach for Natural Language to SQL Translation
    Y Fan, Z He, T Ren, D Guo, L Chen, R Zhu, G Chen, Y Jing, K Zhang, XS Wang
    [Paper, Code]

Education

  • M.S. Computer Science Department Fudan University (09/2022 - 06/2025 Expected)
  • B.S. Computer Science Department Fudan University (09/2018 - 06/2022)