Natural language processing (NLP) is a crucial technology in the era of information age. Exciting advancements in natural language processing (NLP) have recently emerged, enabling systems that can perform tasks such as text translation, question answering, and spoken conversations. This course aims to provide students with a foundational understanding of NLP, including standard frameworks, algorithms, and techniques used to solve various NLP problems. The curriculum will cover topics like language modeling, representation learning, text classification, sequence tagging, syntactic parsing, machine translation, and question answering, with a particular focus on recent deep learning approaches. Through this course, students will receive a comprehensive introduction to NLP concepts, methods, algorithms, applications and state-of-the-art methods research in deep learning for NLP.
Course Content
This course contains; Introduction to Natural Language Processing (NLP),Lingustic Essentials, Regular Exp., Text Normalization, Edit Distance,N-gram Models,Machine Learning Basics, Text Classification, Naive Bayes and Logistic Regression,Vector Semantics and Dense Word Embeddings,Neural Networks and Neural Language Models,Sequence Labeling for Parts of Speech and Named Entities,Exam Week,RNNs and LSTMs,Transformers and Pretrained Language Models, Fine Tuning and Masked Language Models,Machine Translation, Question Answering and Information Retrieval,Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech,Context-Free Grammars, Constituency Parsing, Dependency Parsing, Logical Representations of Sentence Meaning,Review and Project Presentations.
Dersin Öğrenme Kazanımları
Teaching Methods
Assessment Methods
Decompose a real-world problem into subproblems in NLP, use existing natural language processing tools to conduct basic NLP, and identify potential solutions.
11
A, F
Learn about the main uses of machine learning techniques and deep learning models in NLP.
A, F, G
Explain state-of-the-art methods to tackle NLP sub-problems, such as text representation, representation learning techniques, text mining, language modeling, and similarity detection, and gain a an understanding about the methods and metrics for various natural language processing tasks and applications.
A, F, G
Extract information from text automatically using concepts and methods from natural language processing (NLP) including stemming, n-grams, POS tagging, and parsing.
A, E, F
Get familiarized with the terminology, a breadth of concepts and tasks in NLP.
A, G
Teaching Methods:
11: Demonstration Method
Assessment Methods:
A: Traditional Written Exam, E: Homework, F: Project Task, G: Quiz
Course Outline
Order
Subjects
Preliminary Work
1
Introduction to Natural Language Processing (NLP)
Ch 1
2
Lingustic Essentials, Regular Exp., Text Normalization, Edit Distance
Ch 2
3
N-gram Models
Ch 3
4
Machine Learning Basics, Text Classification, Naive Bayes and Logistic Regression
Ch 4, 5
5
Vector Semantics and Dense Word Embeddings
Ch 6
6
Neural Networks and Neural Language Models
Ch 7
7
Sequence Labeling for Parts of Speech and Named Entities
Ch 8
8
Exam Week
Ch 1-8
9
RNNs and LSTMs
Ch 9
10
Transformers and Pretrained Language Models, Fine Tuning and Masked Language Models
Ch 10, 11
11
Machine Translation, Question Answering and Information Retrieval
Ch 13, 14
12
Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech
Sohbet Robotları ve Diyalog Sistemleri, Otomatik Konuşma Tanıma ve Metinden Konuşmaya
13
Context-Free Grammars, Constituency Parsing, Dependency Parsing, Logical Representations of Sentence Meaning
Ch 17, 18, 19
14
Review and Project Presentations
Resources
- Speech and Language Processing, D.Jurafsky, J.H.Martin, 3rd Edition, Pearson-Prentice Hall.
- Foundations of Statistical Natural Language Processing, C.D.Manning, H.Schütze, MIT Press, 2002.
- Jacob Eisenstein, Introduction to Natural Language Processing, 2019.
- Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning
- Delip Rao and Brian McMahan. Natural Language Processing with PyTorch
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers
Course Contribution to Program Qualifications
Course Contribution to Program Qualifications
No
Program Qualification
Contribution Level
1
2
3
4
5
1
1. An ability to apply knowledge of mathematics, science, and engineering
X
2
2. An ability to identify, formulate, and solve engineering problems
X
3
3. An ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability
X
4
4. An ability to use the techniques, skills, and modern engineering tools necessary for engineering practice
X
5
5. An ability to design and conduct experiments, as well as to analyze and interpret data
X
6
6. An ability to function on multidisciplinary teams
X
7
7. An ability to communicate effectively
X
8
8. A recognition of the need for, and an ability to engage in life-long learning
X
9
9. An understanding of professional and ethical responsibility
X
10
10. A knowledge of contemporary issues
11
11. The broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context
X
Assessment Methods
Contribution Level
Absolute Evaluation
Rate of Midterm Exam to Success
30
Rate of Final Exam to Success
70
Total
100
ECTS / Workload Table
Activities
Number of
Duration(Hour)
Total Workload(Hour)
Course Hours
14
3
42
Guided Problem Solving
0
0
0
Resolution of Homework Problems and Submission as a Report
6
12
72
Term Project
0
0
0
Presentation of Project / Seminar
2
8
16
Quiz
0
0
0
Midterm Exam
2
10
20
General Exam
3
10
30
Performance Task, Maintenance Plan
0
0
0
Total Workload(Hour)
180
Dersin AKTS Kredisi = Toplam İş Yükü (Saat)/30*=(180/30)
6
ECTS of the course: 30 hours of work is counted as 1 ECTS credit.
Detail Informations of the Course
Course Description
Course
Code
Semester
T+P (Hour)
Credit
ECTS
INTRODUCTION to NATURAL LANGUAGE PROCESSING
-
Spring Semester
3+0
3
6
Course Program
Prerequisites Courses
Recommended Elective Courses
Language of Course
English
Course Level
First Cycle (Bachelor's Degree)
Course Type
Elective
Course Coordinator
Prof.Dr. Selim AKYOKUŞ
Name of Lecturer(s)
Prof.Dr. Selim AKYOKUŞ
Assistant(s)
Aim
Natural language processing (NLP) is a crucial technology in the era of information age. Exciting advancements in natural language processing (NLP) have recently emerged, enabling systems that can perform tasks such as text translation, question answering, and spoken conversations. This course aims to provide students with a foundational understanding of NLP, including standard frameworks, algorithms, and techniques used to solve various NLP problems. The curriculum will cover topics like language modeling, representation learning, text classification, sequence tagging, syntactic parsing, machine translation, and question answering, with a particular focus on recent deep learning approaches. Through this course, students will receive a comprehensive introduction to NLP concepts, methods, algorithms, applications and state-of-the-art methods research in deep learning for NLP.
Course Content
This course contains; Introduction to Natural Language Processing (NLP),Lingustic Essentials, Regular Exp., Text Normalization, Edit Distance,N-gram Models,Machine Learning Basics, Text Classification, Naive Bayes and Logistic Regression,Vector Semantics and Dense Word Embeddings,Neural Networks and Neural Language Models,Sequence Labeling for Parts of Speech and Named Entities,Exam Week,RNNs and LSTMs,Transformers and Pretrained Language Models, Fine Tuning and Masked Language Models,Machine Translation, Question Answering and Information Retrieval,Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech,Context-Free Grammars, Constituency Parsing, Dependency Parsing, Logical Representations of Sentence Meaning,Review and Project Presentations.
Dersin Öğrenme Kazanımları
Teaching Methods
Assessment Methods
Decompose a real-world problem into subproblems in NLP, use existing natural language processing tools to conduct basic NLP, and identify potential solutions.
11
A, F
Learn about the main uses of machine learning techniques and deep learning models in NLP.
A, F, G
Explain state-of-the-art methods to tackle NLP sub-problems, such as text representation, representation learning techniques, text mining, language modeling, and similarity detection, and gain a an understanding about the methods and metrics for various natural language processing tasks and applications.
A, F, G
Extract information from text automatically using concepts and methods from natural language processing (NLP) including stemming, n-grams, POS tagging, and parsing.
A, E, F
Get familiarized with the terminology, a breadth of concepts and tasks in NLP.
A, G
Teaching Methods:
11: Demonstration Method
Assessment Methods:
A: Traditional Written Exam, E: Homework, F: Project Task, G: Quiz
Course Outline
Order
Subjects
Preliminary Work
1
Introduction to Natural Language Processing (NLP)
Ch 1
2
Lingustic Essentials, Regular Exp., Text Normalization, Edit Distance
Ch 2
3
N-gram Models
Ch 3
4
Machine Learning Basics, Text Classification, Naive Bayes and Logistic Regression
Ch 4, 5
5
Vector Semantics and Dense Word Embeddings
Ch 6
6
Neural Networks and Neural Language Models
Ch 7
7
Sequence Labeling for Parts of Speech and Named Entities
Ch 8
8
Exam Week
Ch 1-8
9
RNNs and LSTMs
Ch 9
10
Transformers and Pretrained Language Models, Fine Tuning and Masked Language Models
Ch 10, 11
11
Machine Translation, Question Answering and Information Retrieval
Ch 13, 14
12
Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech
Sohbet Robotları ve Diyalog Sistemleri, Otomatik Konuşma Tanıma ve Metinden Konuşmaya
13
Context-Free Grammars, Constituency Parsing, Dependency Parsing, Logical Representations of Sentence Meaning
Ch 17, 18, 19
14
Review and Project Presentations
Resources
- Speech and Language Processing, D.Jurafsky, J.H.Martin, 3rd Edition, Pearson-Prentice Hall.
- Foundations of Statistical Natural Language Processing, C.D.Manning, H.Schütze, MIT Press, 2002.
- Jacob Eisenstein, Introduction to Natural Language Processing, 2019.
- Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning
- Delip Rao and Brian McMahan. Natural Language Processing with PyTorch
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers
Course Contribution to Program Qualifications
Course Contribution to Program Qualifications
No
Program Qualification
Contribution Level
1
2
3
4
5
1
1. An ability to apply knowledge of mathematics, science, and engineering
X
2
2. An ability to identify, formulate, and solve engineering problems
X
3
3. An ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability
X
4
4. An ability to use the techniques, skills, and modern engineering tools necessary for engineering practice
X
5
5. An ability to design and conduct experiments, as well as to analyze and interpret data
X
6
6. An ability to function on multidisciplinary teams
X
7
7. An ability to communicate effectively
X
8
8. A recognition of the need for, and an ability to engage in life-long learning
X
9
9. An understanding of professional and ethical responsibility
X
10
10. A knowledge of contemporary issues
11
11. The broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context
