Implementing a Large Language Model From Scratch
Author: Kirill Dragunov | Data Science
Abstract:
Large Language Models (LLMs) have transformed not only modern artificial intelligence, but also many aspects of professional and academic life. Today, they are widely used by students to prepare for classes, exams, and projects, as well as by practicing professionals, who use LLMs to increase their productivity by assisting with certain aspects of their work.
To understand how LLMs achieve the seemingly “magical” ability to turn simple or complex prompts into coherent, multi-paragraph responses, one of the most effective approaches is to try and build one from the ground up.
This poster presents the implementation of a minimal yet functional transformer-based architecture, covering every stage of the pipeline: text preprocessing and tokenization, embedding construction, attention mechanisms, and the training process. By reconstructing these components from first principles, it is aimed at providing a conceptual and practical understanding of how modern LLMs operate.
Graduate Poster Exhibition at SPARK!
The Graduate Poster Exhibition celebrates the research and creative work of the graduate community, showcasing everything from prose and code to original research and artistic expression. As part of SPARK! (Showcase of Projects, Art, Research, and Knowledge), a reimagining of Research Week, the exhibition highlights the depth, range, and impact of graduate scholarship and invites the campus community to engage with ideas taking shape across disciplines.
Mastering Data: The Data Science Program
The Master of Science in Data Science at Rutgers University–Camden is an interdisciplinary program that equips students with skills in data analysis, machine learning, and statistical modeling. The curriculum integrates tools from mathematics, computer science, public policy, and prevention science, emphasizing practical applications across various industries. Core courses include Foundations of Data Science, Statistical Methods for Data Science, Data Visualization, and Applied Data Mining and Machine Learning. Students also engage in a thesis or capstone project, applying their knowledge to real-world problems.
Spotlights & Stories
Want more stories like this? Spotlights and Stories highlights alumni, faculty, staff, and student experiences through stories, video, and voice. Learn more
Explore Graduate School Programs
Rutgers Graduate School-Camden offers 20+ graduate certificates, master’s, and doctoral programs across various disciplines, including biology, data science, creative writing, and psychology. We take pride in our academic diversity. Learn more
Former Students, Get Involved
Stay connected with Rutgers-Camden! Engage with former students, attend events, and support current students. Your involvement strengthens our community and helps shape future success. Learn more