About this course
Build applications on large language models and autonomous agents, covering transformer theory, prompt engineering, retrieval-augmented generation, tool use, multi-agent orchestration, evaluation, safety, and parameter-efficient fine-tuning.
Course format. Thirteen weeks, four contact hours each: a two-hour lecture (concepts and theory) and a two-hour practice session. The course is project-based; teams carry one running project end to end and present it three times, in weeks 5, 8, and 13.
What you will build
Teams build a domain-specific LLM application that progresses from a zero-shot API baseline (OpenAI and Anthropic APIs) through an engineered RAG pipeline (LlamaIndex, FAISS, Qdrant) to a multi-step agentic workflow with tool use (LangChain), Langfuse observability, a structured RAGAS evaluation suite, Guardrails AI safety controls, and an OWASP LLM risk review.
Expected outcomes
- Derive the transformer architecture from multi-head self-attention, Q/K/V projections, positional encoding, and feed-forward sublayers, and explain causal language modeling as autoregressive next-token prediction optimized by cross-entropy
- Apply prompt engineering techniques — zero-shot, few-shot, chain-of-thought, ReAct, self-consistency, and meta-prompting — using the OpenAI and Anthropic APIs, and analyze how in-context learning and emergent capabilities steer conditional token distributions
- Build retrieval-augmented generation pipelines with dense embedding retrieval and context injection, and extend them with advanced techniques including hybrid search, re-ranking, HyDE, and multi-query retrieval using LlamaIndex, FAISS, and Qdrant
- Design multi-step autonomous agents with tool use, function calling, episodic and semantic memory, and multi-agent orchestration via LangChain, and assess agent reliability with trajectory evaluation and benchmark suites such as GAIA and SWE-bench
- Evaluate LLM systems with offline benchmarks, RAG-specific RAGAS metrics (faithfulness, context precision, answer relevance), and assertion-based pipeline testing, and harden deployments against hallucination and the OWASP LLM Top 10 using Guardrails AI
- Apply LoRA and QLoRA parameter-efficient fine-tuning with Hugging Face PEFT, and instrument production serving economics with KV-cache management, quantization, dynamic batching, and Langfuse cost-per-token dashboards
Key topics
- LLM fundamentals
- Prompting & RAG
- Tool use & agents
- Evaluation & guardrails
Theoretical foundations
The concepts and results this course rests on.
- transformer architecture: multi-head self-attention with Q/K/V projections, positional encoding, and feed-forward sublayers with residual connections and layer normalization
- causal language modeling: autoregressive factorization, next-token prediction, teacher forcing, and perplexity as cross-entropy
- emergent capabilities and in-context learning: few-shot examples in context, capability threshold behavior, and chain-of-thought prompting
- retrieval-augmented generation: chunking strategies, dense embedding generation, approximate nearest-neighbor indexing, and context injection
- advanced RAG: hybrid sparse-dense search, cross-encoder re-ranking, hypothetical document embeddings (HyDE), and multi-query retrieval
- prompt engineering: zero-shot, few-shot, system/role instructions, ReAct, self-consistency sampling, and meta-prompting
- LLM agent architectures: tool use and function calling, episodic and semantic memory, agent orchestration frameworks, and trajectory evaluation with benchmark suites
- LLM evaluation: offline benchmarks (MMLU, HumanEval), RAG-specific metrics (faithfulness, context precision, answer relevance), assertion-based pipeline testing, and human preference evaluation
- hallucination, grounding, and safety: factuality controls, output validation, RLHF alignment, and OWASP LLM Top 10 risk taxonomy
- parameter-efficient fine-tuning: LoRA and QLoRA rank-decomposition adapters, instruction-dataset curation; and inference economics: KV-cache management, post-training quantization, dynamic batching, and cost-per-token accounting
Prerequisites
This is a Year-3 course. It assumes the mandatory CS core: data structures and algorithms, operating systems, computer networks, databases, software engineering, and the core mathematics (linear algebra, probability and statistics, calculus, discrete mathematics). It additionally requires the specific prior courses listed below.
Course-specific prerequisites:
- Machine Learning and Deep Learning
- Probability and linear algebra
- Python
Weekly schedule 13 weeks · lecture + practice
Foundations
Wk 1
From language modeling to transformers
LectureWe formalize the language modeling objective, the chain rule of probability over tokens, and tokenization (P2), then introduce the transformer — multi-head self-attention, Q/K/V projections, positional encoding, and feed-forward sublayers (P1).
PracticeOpenAI API: set up the development environment, call the OpenAI Chat Completions API, and run zero-shot, greedy, and sampled generation on a domain task.
ProjectInitialize the team project repo and stand up a zero-shot API baseline using the OpenAI API over the chosen domain.
Wk 2
Attention mechanics and multi-model comparison
LectureWe derive scaled dot-product attention, multi-head attention, the full encoder-decoder block with residuals and layer norm (P1), and survey how frontier models differ in capability profiles and API conventions.
PracticeAnthropic API: call the Anthropic Messages API side-by-side with OpenAI; compare outputs, token costs, and latency on the same prompt set.
ProjectAdd a multi-model comparison layer: run baseline prompts through both OpenAI and Anthropic APIs and log response quality differences.
Wk 3
Training, scaling, and local inference
LectureWe cover pretraining with cross-entropy loss and backpropagation through the transformer (P2), scaling laws relating loss to parameters, data, and compute, and emergent capability threshold behavior (P3).
PracticeOllama: serve a local open-weight model with Ollama; compare latency and output quality against the hosted APIs; use Ollama for fast, cost-free iteration in development.
ProjectAdd a local inference fallback with Ollama for offline development and cost-free prompt iteration.
Prompting and retrieval
Wk 4
Prompt engineering and in-context learning
LectureWe analyze in-context learning, few-shot demonstrations, chain-of-thought reasoning, ReAct, self-consistency sampling, and meta-prompting, and the theory of why prompts steer conditional token distributions (P3, P6).
PracticeOpenAI API: build a systematic prompt-template library and run A/B evaluation across zero-shot, few-shot, chain-of-thought, and self-consistency variants on a held-out task set.
ProjectEngineer a structured prompt layer with few-shot exemplars and chain-of-thought instructions that measurably improves answer quality over the zero-shot baseline.
Wk 5
Embeddings and vector retrievalPresentation
LectureWe study dense embeddings, cosine similarity, approximate nearest-neighbor indexing, chunking bias-variance trade-offs, and the basic RAG formulation of marginalizing over retrieved passages (P4).
PracticeFAISS: encode the domain corpus with an embedding model and build a FAISS flat index; measure retrieval recall at k. Presentation 1: each team defends its project specification, target domain, and evaluation plan.
ProjectLock the project specification; deliver Presentation 1; stand up a FAISS embedding index over the project knowledge base.
Wk 6
Advanced RAG and production vector stores
LectureWe cover hybrid search combining sparse BM25 and dense retrieval, cross-encoder re-ranking, hypothetical document embeddings (HyDE), multi-query retrieval, and agentic RAG patterns (P5); we also revisit grounding faithfulness metrics (P4).
PracticeLlamaIndex and Qdrant: build a full RAG pipeline with LlamaIndex using Qdrant as the vector backend; add hybrid search and re-ranking and compare retrieval quality against the FAISS baseline.
ProjectConvert the project to a grounded RAG assistant using LlamaIndex over Qdrant; demonstrate measurable faithfulness improvement over the zero-shot baseline.
Agents and tools
Wk 7
Tool use, function calling, and orchestration
LectureWe cover structured output, JSON schemas, function calling, tool result injection, and how tool access extends a model beyond its parametric knowledge; we introduce multi-step task decomposition and agent memory types (P7).
PracticeLangChain: define tool schemas for calculator, search, and code-execution tools; wire them into a LangChain agent; observe and debug the tool-call trace.
ProjectGive the assistant tool access via LangChain; demonstrate a complete multi-step task executed end to end with logged tool calls.
Wk 8
Agentic loops and multi-agent systemsPresentation
LectureWe derive the ReAct reasoning-acting loop, reasoning-acting interleaving, episodic and semantic memory, multi-agent coordination patterns, and benchmark evaluation suites for agents (GAIA, SWE-bench, WebArena) (P7).
PracticePresentation 2: interim demo — each team demonstrates the agent executing a multi-step goal with a tool-using ReAct loop; peers ask questions and the team defends design choices.
ProjectDeliver Presentation 2; demonstrate the agent planning, selecting tools, and chaining calls toward a goal using the ReAct loop.
Evaluation and guardrails
Wk 9
Evaluation of LLM and RAG systems
LectureWe cover offline benchmarks (MMLU, HumanEval), RAG-specific evaluation metrics (faithfulness, context precision, answer relevance), assertion-based pipeline testing, step-level trajectory logging, and human preference evaluation for alignment (P8).
PracticeGuardrails AI: define output schemas and build assertion-based guards; run an automated evaluation harness that scores every agent response against a labeled held-out set.
ProjectBuild a RAGAS evaluation suite measuring faithfulness, context precision, and answer relevance; integrate it as a regression check that runs on every project change.
Wk 10
Observability and cost accounting
LectureWe cover KV-cache utilization, dynamic batching, latency-throughput trade-offs, token cost modeling, and production tracing and logging for LLM pipelines (P10); we also review faithfulness of tool-selection chains as an evaluation dimension (P8).
PracticeLangfuse: instrument every LLM call with Langfuse tracing; surface per-query token counts, latency, and cost; use trace data to identify the slowest and most expensive pipeline steps.
ProjectIntegrate Langfuse observability into the full pipeline; produce a cost-per-query dashboard and identify at least one optimization based on the trace data.
Wk 11
Hallucination, safety, and OWASP LLM risks
LectureWe cover factuality controls, output validation, RLHF alignment and preference optimization concepts, prompt injection, jailbreaks, content filtering, and the OWASP LLM Top 10 risk taxonomy with mitigations (P9).
PracticeGuardrails AI: add structured output validation, prompt injection detection, and content filtering guardrails; red-team the agent for injection and jailbreak scenarios.
ProjectHarden the agent with Guardrails AI safety controls; document and remediate the top OWASP LLM risks for the application domain.
Wk 12
Parameter-efficient fine-tuning and inference economics
LectureWe cover LoRA and QLoRA rank-decomposition adapters, instruction-dataset curation, adapter merging, post-training quantization, KV-cache management, dynamic batching, and cost-per-token production serving strategies (P10).
PracticePEFT (Hugging Face): fine-tune a small open-weight model with LoRA using Hugging Face PEFT on a domain instruction dataset; measure the accuracy gain and inference latency against the base model.
ProjectAdd a LoRA-fine-tuned adapter for the specialized domain task; compare cost and accuracy against the RAG-only baseline and document the trade-offs.
Capstone
Wk 13
Final defensePresentation
LectureWe synthesize the full stack — from transformer attention theory to deployed agentic systems — and survey open research directions in LLM reasoning, alignment, and agent reliability.
PracticePresentation 3: final demo with live RAGAS evaluation results, Langfuse cost dashboard, and an oral defense of all major design choices including RAG strategy, agent architecture, safety controls, and fine-tuning decisions.
ProjectDeliver the complete domain-specific LLM application: zero-shot baseline through engineered RAG pipeline to multi-step agentic workflow, with Langfuse observability, RAGAS evaluation suite, Guardrails AI safety controls, and OWASP LLM risk review.
AI tools in this course.
Students lean on AI coding assistants throughout, generating and refactoring LangChain agent wiring, LlamaIndex retrieval pipelines, and FAISS/Qdrant indexing code, then building iteratively from a bare API call to a full agentic workflow. They use AI to scaffold ReAct tool definitions and function-calling schemas, to synthesize few-shot exemplars and labeled evaluation sets, and to draft RAGAS rubrics and Guardrails AI validator specs. AI assistants also help interpret Langfuse tracing logs and red-team transcripts, turning raw eval output into diagnoses of why an agent hallucinated or failed an injection test.
Student project
Teams build a domain-specific LLM application that progresses from a zero-shot API baseline through an engineered RAG pipeline to a multi-step agentic workflow with tool use. Each stage adds measurable capability backed by the theory taught that week. The final deliverable includes Langfuse observability, a structured RAGAS evaluation suite, Guardrails AI safety controls, and an OWASP LLM risk review.
Requirements
- Build a working system, not a set of disconnected exercises.
- Be original: a new system that solves a real problem, not a re-implementation of a tutorial or course demo.
- Show real depth: real data, real users or realistic load, and engineering trade-offs that are measured rather than assumed.
- Carry one running project from specification to a deployed, defensible result across the whole term.
- Work in a team of three or four and defend the design at each of the three presentations (weeks 5, 8, and 13).
Example projects
Research-paper Q and A assistantCodebase navigator agentCustomer-support RAG botLegal or policy document advisorPersonal data analyst agentTravel-planning tool agentMedical-literature triage assistantFinancial-report summarizer
Assessment & grading
Grading is project-based, with no written exam. Teams of three or four present one running project three times.
| Component | What it covers | Weight |
|---|---|---|
| Project · Specification | Presentation 1 (week 5): problem, objectives, and architecture | 20% |
| Project · Interim | Presentation 2 (week 8): the working system demonstrated live | 30% |
| Project · Final | Presentation 3 (week 13): end-to-end demo with oral defense | 50% |
Tools & platforms
- OpenAI API: hosted frontier models, function calling, and zero-shot baseline
- Anthropic API: multi-model comparison and Claude-based reasoning
- LangChain: agent orchestration, tool wiring, and ReAct agent loops
- LlamaIndex: RAG pipeline construction, retrieval, and indexing
- FAISS: approximate nearest-neighbor vector search
- Qdrant: production vector database with hybrid search support
- Langfuse: LLM observability, tracing, and cost-per-token dashboards
- PEFT (Hugging Face): LoRA and QLoRA parameter-efficient fine-tuning
- Guardrails AI: structured output validation, injection detection, and safety filtering
- Ollama: local open-weight model serving for development
Free online courses
Existing free, video-based courses this course can build on, for self-study or as a teaching basis.
- YouTubeStanford CS224N: NLP with Deep Learning (Spring 2024)
- YouTubeStanford CS25: Transformers United (V6)
In Hebrew · בעברית
- Dr. Amos Azaria, Ariel University (YouTube)Deep Learning and NLP - קורס למידה עמוקה ועיבוד שפות טבעיות
- Yoav Goldberg / Ido Dagan (YouTube)עיבוד שפות טבעיות (Natural Language Processing)
- Google Cloud (Coursera)Transformer Models and BERT Model - בעברית
Primary literature
Seminal works for advanced study.
References
Books and resources link to an online or publisher page.
- PaperAttention Is All You Need
- PaperBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- PaperRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- PaperReAct: Synergizing Reasoning and Acting in Language Models
- PaperScaling Laws for Neural Language Models
- TextbookSpeech and Language Processing, 3rd edition draft
- TextbookDive into Deep Learning
- DocumentationHugging Face Transformers Documentation
Role in each concentration
| Concentration | Role |
|---|---|
| Intelligent Software Systems | Core · Semester 1 |
| Networking & Cyber Security | Elective |
| AI & Robotics | Core · Semester 1 |
| AI and Quantum Computing for Finance | Core · Semester 1 |
| Immersive Systems & Game Development | Elective |
| Defense Technologies & Autonomous Systems | Elective |