Using Open Source AI Models with Hugging Face | Build Free AI Models

Alara, a PhD candidate at Imperial College London and a former machine learning engineer at Hugging Face, introduces a code along focused on using open source AI models with the Hugging Face project. She discusses Hugging Face's mission to democratize AI research through open source tools and libraries, highlighting their platform, The Hub, which functions similarly to GitHub for model and data set repositories. Alara emphasizes the ease of finding and using models and data sets on The Hub and mentions the additional resources like the Transformers library, datasets, and diffusers. She sets the stage for the code along, which will teach how to create custom machine learning pipelines for tasks like multilingual text translation and image captioning, using the Hugging Face ecosystem.

The second paragraph delves into the specifics of loading pretrained models from the Hugging Face Hub using the Transformers library. Alara explains how the Hub serves as a git platform for model checkpoints and data sets, and how the Transformers library is integrated with it. She discusses the utility of auto classes like Auto Model and Auto Tokenizer for easily loading models and their associated preprocessing tools by simply providing the repository name. Alara also touches on the structure of model repositories on the Hub and demonstrates how to use the from_pretrained method to load a text classification model trained to predict emoji labels from tweets.

In this segment, Alara discusses the role of tokenizers in natural language processing (NLP), explaining how they convert text into a format that machine learning models can understand. She demonstrates the creation of a tokenizer object using the RobertaTokenizerFast class and discusses its capabilities, such as handling padding, truncation, and unknown words. Alara then proceeds to load a pretrained model using the repository name and the from_pretrained method, noting the importance of matching the correct AutoModel class to the model's task, such as RobertaForSequenceClassification for the emoji classification model. She also mentions how to identify the exact class and task of a model using its configuration.

The focus of the fourth paragraph is on building NLP pipelines for text translation using the flan-T5 base model by Google. Alara describes flan-T5 as a powerful multilingual translation model that can perform various text-related tasks. She outlines the process of preparing input text for the model, emphasizing the need to specify source and target languages for multilingual models. Alara demonstrates how to use the tokenizer to preprocess the input text and convert it into token IDs and attention masks, which are then fed into the model for inference. The paragraph concludes with a discussion on performing inference and generating human-readable translations from the model's token ID outputs.

Alara introduces the Hugging Face Datasets library, which simplifies the process of finding and loading datasets from the Hub. She provides an overview of a fashion image captioning dataset that will be used in the code along, explaining its structure and the ease with which it can be loaded into a dataset dictionary. The paragraph also covers how to access specific subsets of a dataset and how to work with the data samples, including visualizing images and handling their corresponding text captions.

This paragraph introduces the process of building an image captioning pipeline using the BLIP model by Salesforce. Alara contrasts BLIP with language models like flan-T5, highlighting that BLIP is a multimodal model designed for conditional generation tasks. She demonstrates how to initialize the BLIP processor and model, preprocess images from the dataset, and perform inference to generate captions. Alara also discusses the output format of BLIP and how to decode the token IDs into human-readable captions, providing an example of generating a new caption for a given image.

The final paragraph covers the process of creating a mapping function to preprocess and generate new captions for all samples in the dataset. Alara explains the mapping functionality of the datasets library, which allows applying a function to all data samples efficiently. She demonstrates how to create a utility function named 'replace_caption' and apply it to the entire dataset using the map method. The paragraph concludes with a discussion on pushing the updated dataset to the Hugging Face Hub, detailing the steps required to log in to the Hub, and use the push_to_hub method to upload the dataset, making it accessible for further use and experimentation.