Mastering Transformers

📖 مسلط شدن بر ترنسفورمرها – از BERT تا GPT در NLP، CV و مولتی‌مودال

🧠 مدل‌های زبانی مبتنی بر ترنسفورمر مثل BERT، T5، GPT، DALL‑E و ChatGPT حالا دیگه سلطان بی‌رقیب NLP هستن و تبدیل شدن به پارادایم جدید. دلیلش هم واضحه: فاین‌تیون سریع و دقیق که به راحتی مدل‌های کلاسیک ML رو توی اکثر چالش‌های NLU شکست می‌ده.

ولی ماجرا فقط به متن محدود نیست! 🚀 امروز حوزه مولتی‌مودال و Generative AI هم با ترنسفورمرها غوغا کرده، از Text‑to‑Image تا Vision Transformers.

🎯 ویژگی‌های کلیدی

• 📐 شناخت کامل معماری‌های پیچیده Deep Learning و ترنسفورمر.
• 🛠 حل مسائل NLP و Computer Vision صنعتی با رویکرد عملی.
• 🗂 نکات کلیدی در آماده‌سازی داده‌ها و سازگارکردن دیتاست با زبان یا مسئله خاص.

📝 توضیحات کلی کتاب

📚 این کتاب از مدل‌های پایه شروع می‌کنه و می‌رسه به آموزش و فاین‌تیون مدل‌های زبان خودرگرسیو (Autoregressive) مثل GPT و XLNet. بعدش می‌ری سراغ بهینه‌سازی عملکرد، Monitoring با TensorBoard و در نهایت استفاده از Vision Transformers برای حل مسائل CV.

💡 حتی یاد می‌گیری چطور با ترنسفورمرها مدل‌های سری زمانی بسازی و پیش‌بینی انجام بدی.

🔍 چیزهایی که یاد می‌گیری

• 🐍 حل مسائل NLP از ساده تا پیچیده با پایتون.
• 📊 پیاده‌سازی Classification/Regression هم با روش‌های سنتی و هم با ترنسفورمر.
• 🎯 آموزش و فاین‌تیون مدل‌های زبان روی تسک‌های Downstream.
• 🌐 استفاده از ترنسفورمر در Generative AI و Computer Vision.
• 💻 ساخت اپ‌های NLP با کتابخانه transformers پایتون.
• 🗣 تمرکز روی تولید زبان (مثل ترجمه ماشینی و Chatbot چندزبانه).
• ⚡ کاهش Latency در Inference مدل‌ها.

📂 فهرست فصل‌ها

بخش ۱ – تازه‌های فیلد، نصب‌ها و Hello World

1. از Bag‑of‑Words تا ترنسفورمر
2. مقدمه عملی (Hands‑on) بر موضوع

بخش ۲ – مدل‌های ترنسفورمر: از Autoencoder تا Autoregressive

1. مدل‌های زبانی Autoencoder
2. از مدل‌های مولد تا LLMها
3. فاین‌تیون برای Text Classification
4. فاین‌تیون برای Token Classification
5. نمایش برداری متن (Text Representation)
6. ارتقای عملکرد مدل‌ها (Boosting)
7. فاین‌تیون بهینه با پارامتر کمتر

بخش ۳ – مباحث پیشرفته

1. مدل‌های زبانی بزرگ (LLMها)
2. XAI در NLP
3. ترنسفورمرهای بهینه و سبک
4. مدل‌سازی چندزبانه و Cross‑Lingual
5. Deploy و سرویس‌دهی مدل‌ها
6. ردیابی و مانیتورینگ مدل

بخش ۴ – فراتر از NLP

1. Vision Transformers
2. مولتی‌مودال و Generative Transformers
3. بازنگری معماری‌ها برای سری‌های زمانی

👥 نویسندگان

ساواش ییلدریم – دکترای NLP، دانشیار دانشگاه بیلگی استانبول، محقق مهمان دانشگاه رایرسون کانادا، با ۲۰+ سال تجربه تدریس و توسعه نرم‌افزارهای متن‌باز در حوزه NLP.

میثم عسگری‑چناه‌لو – مدیر AI در Carbon Consulting و دکترای در حال انجام در دانشگاه تبریز، با سابقه پروژه‌های بزرگ در NLU و جستجوی معنایی برای شرکت‌های مخابراتی و بانکی ترکیه.

Explore transformer-based language models from BERT to GPT, delving into NLP and computer vision tasks, while tackling challenges effectively

Key Features:

• Understand the complexity of deep learning architecture and transformers architecture
• Create solutions to industrial natural language processing (NLP) and computer vision (CV) problems
• Explore challenges in the preparation process, such as problem and language-specific dataset transformation

Book Description:

Transformer-based language models such as BERT, T5, GPT, DALL-E, and ChatGPT have dominated NLP studies and become a new paradigm. Thanks to their accurate and fast fine-tuning capabilities, transformer-based language models have been able to outperform traditional machine learning-based approaches for many challenging natural language understanding (NLU) problems.

Aside from NLP, a fast-growing area in multimodal learning and generative AI has recently been established, showing promising results. Mastering Transformers will help you understand and implement multimodal solutions, including text-to-image. Computer vision solutions that are based on transformers are also explained in the book. You’ll get started by understanding various transformer models before learning how to train different autoregressive language models such as GPT and XLNet. The book will also get you up to speed with boosting model performance, as well as tracking model training using the TensorBoard toolkit. In the later chapters, you’ll focus on using vision transformers to solve computer vision problems. Finally, you’ll discover how to harness the power of transformers to model time series data and for predicting.

By the end of this transformers book, you’ll have an understanding of transformer models and how to use them to solve challenges in NLP and CV.

What You Will Learn:

• Focus on solving simple-to-complex NLP problems with Python
• Discover how to solve classification/regression problems with traditional NLP approaches
• Train a language model and explore how to fine-tune models to the downstream tasks
• Understand how to use transformers for generative AI and computer vision tasks
• Build transformer-based NLP apps with the Python transformers library
• Focus on language generation such as machine translation and conversational AI in any language
• Speed up transformer model inference to reduce latency

Table of Contents

Part 1: Recent Developments in the Field, Installations, and Hello World Applications

Chapter 1: From Bag-of-Words to the Transformers

Chapter 2: A Hands-On Introduction to the Subject

Part 2: Transformer Models: From Autoencoders to Autoregressive Models

Chapter 3: Autoencoding Language Models

Chapter 4: From Generative Models to Large Language Models

Chapter 5: Fine-Tuning Language Models for Text Classification

Chapter 6: Fine-Tuning Language Models for Token Classification

Chapter 7: Text Representation

Chapter 8: Boosting Model Performance

Chapter 9: Parameter Efficient Fine-Tuning

Part 3: Advanced Topics

Chapter 10: Large Language Models

Chapter 11: Explainable AI (XAI) in NLP

Chapter 12: Working with Efficient Transformers

Chapter 13: Cross-Lingual and Multilingual Language Modeling

Chapter 14: Serving Transformer Models

Chapter 15: Model Tracking and Monitoring

Part 4: Transformers beyond NLP

Chapter 16: Vision Transformers

Chapter 17: Multimodal Generative Transformers

Chapter 18: Revisiting Transformers Architecture for Time Series

Who this book is for:

This book is for deep learning researchers, hands-on practitioners, and ML/NLP researchers. Educators, as well as students who have a good command of programming subjects, knowledge in the field of machine learning and artificial intelligence, and who want to develop apps in the field of NLP as well as multimodal tasks will also benefit from this book’s hands-on approach. Knowledge of Python (or any programming language) and machine learning literature, as well as a basic understanding of computer science, are required.

About the Author

Savaş Yıldırım graduated from the Istanbul Technical University Department of Computer Engineering and holds a Ph.D. degree in Natural Language Processing (NLP). Currently, he is an associate professor at the Istanbul Bilgi University, Turkey, and is a visiting researcher at the Ryerson University, Canada. He is a proactive lecturer and researcher with more than 20 years of experience teaching courses on machine learning, deep learning, and NLP. He has significantly contributed to the Turkish NLP community by developing a lot of open source software and resources. He also provides comprehensive consultancy to AI companies on their R&D projects. In his spare time, he writes and directs short films, and enjoys practicing yoga.

Meysam Asgari-Chenaghlu is an AI manager at Carbon Consulting and is also a Ph.D. candidate at the University of Tabriz. He has been a consultant for Turkey’s leading telecommunication and banking companies. He has also worked on various projects, including natural language understanding and semantic search.