Learning Material

Welcome to Generative AI for Software Development (CSCI 455/555). What is GenAI, the evolution of AI for code, course structure, tools of the trade, AI usage policy, ethics & responsible AI, and getting set up for the semester.

How to collect, clean, and prepare source-code data from public repositories. Covers preprocessing, lexer vs BPE tokenization, ASTs, deduplication with hashing, data splitting, ethics & licensing, and data provenance.

Probability refresher, MLE, chain rule, n-gram models, Zipf's law, the naturalness hypothesis, perplexity & cross-entropy, smoothing, temperature & sampling strategies, OOV handling, and the path from n-grams to LLMs.

Classification metrics, BLEU (with worked example), ROUGE, METEOR, CodeBLEU, CrystalBLEU, pass@k for code generation, embeddings primer, cosine similarity, contrastive learning, SIDE framework, human evaluation, and common evaluation pitfalls.

Neural network fundamentals, backpropagation, hyperparameters, non-generative tasks (clone detection, vulnerability prediction), embeddings, LSTMs, GRUs, seq2seq, attention, transformers, autoregressive generation, pre-training vs fine-tuning, and the DL4SE toolkit.

In-Context Learning, few-shot prompting, chain-of-thought, prompt engineering best practices, RAG, tool use & function calling, context window management, prompt chaining, self-consistency, and evaluating prompt effectiveness.

How LLMs generate code, temperature & sampling as a hallucination factor, the CodeHalu taxonomy, spot-the-hallucination exercises, RAG for mitigation, prompt engineering defenses, tool-augmented generation, production case studies, and building hallucination-resistant workflows.

GA fundamentals (selection, crossover, mutation), the evaluation bottleneck in code generation, why CodeBLEU is not enough, fitness approximation with LLM predictors (95% accuracy, 5 orders of magnitude speedup), the GA+LLM architecture, honest limitations, and research frontiers.