ITE 403: Introduction to Natural Language Processing

Course Description

This course introduces students to modern methodologies in natural language processing (NLP) with an emphasis on machine learning and deep learning. Topics include the NLP data pipeline, data acquisition, data visualization, data pre-processing methods, and NLP model deployment. Students implement and analyze common NLP tasks and applications including language detection, transliteration, translation, sentiment analysis, and dialogue systems. (3 credits)

Prerequisite

• ITE 301: Introduction to AI and Generative AI

Student Learning Outcomes (SLOs)

Students who successfully complete this course will be able to:

1. Describe the fundamental concepts, core components, and key features of Natural Language Processing (NLP), and illustrate their use in real-world applications.
2. Describe commonly used NLP datasets and explain methods for text data acquisition and storage.
3. Explain the importance of data curation in NLP and describe key curation practices that influence model performance and reliability.
4. Apply the NLTK library in Python to preprocess and analyze text data, including tokenization, stop-word removal, and basic linguistic analysis.
5. Discuss the importance of data visualization in NLP and apply data visualization techniques specific to NLP.
6. Describe and compare popular text vectorization methods and implement data pre-processing techniques to impact accuracy of LLMs.
7. Explain and apply document similarity techniques, distance metrics, and vector visualization methods for similarity search and clustering in NLP and LLM datasets.
8. Describe the key stages of the NLP data pipeline and explain how the pipeline supports the training and deployment of large language models (LLMs).
9. Describe and apply NLP classifiers, neural networks, and language models, and explain how reinforcement learning from human feedback (RLHF) improves model behavior.
10. Explain and apply various neural language models, including N-gram and sequential models.
11. Explain Recurrent Neural Networks (RNNs) and Named Entity Recognition (NER), and implement them in practical NLP tasks.
12. Describe various machine learning (ML) model deployment platforms and demonstrate ML model deployment using web applications like Streamlit and Flask.
13. Describe key features and components of dialogue systems and chatbots, including AI-powered generative chatbots, and discuss conversational memory and multi-modal chatbots.
14. Design, create and test a Generative AI powered Chatbot using BotPress.
15. Implement NLP tasks such as language detection, transliteration, translation, and sentiment analysis across multiple languages.
16. Explain the workings of LSTM, Transformers, and ChatGPT, and examine features of various pretrained generative NLP models.
17. Compare Generative NLP models, including PaLM, PaLM 2, Med-PaLM 2, LaMDA, and LLaMA, by examining their architectures, parameters, and performance on specific use cases.
18. Explain the process of building a large language model (LLM), explore its components using LangChain, and discuss the practical implementation and computational requirements in generative AI.

Course Activities and Grading

Required Textbook

• This course uses Open Educational Resources (OER). OER are openly licensed, educational resources that can be used for teaching, learning and research. OER may consist of a variety of resources such as textbooks, videos and software that are no cost for students.

Course Schedule

COSC Accessibility Statement

Charter Oak State College encourages students with disabilities, including non-visible disabilities such as chronic diseases, learning disabilities, head injury, attention deficit/hyperactive disorder, or psychiatric disabilities, to discuss appropriate accommodations with the Office of Accessibility Services at OAS@charteroak.edu.

COSC Policies, Course Policies, Academic Support Services and Resources

Students are responsible for knowing all Charter Oak State College (COSC) institutional policies, course-specific policies, procedures, and available academic support services and resources. Please see COSC Policies for COSC institutional policies, and see also specific policies related to this course. See COSC Resources for information regarding available academic support services and resources.