- Deepspeed is now supported for fine-tuning.
- Apple’s MPS backend is now automatically used for both training and fine-tuning if detected.
- Evaluating data is now used during fine-tuning to track the fine-tuning progress.
- WandB can now be used to log the results from fine-tuning.
- CSV files are supported for training/evaluating text generation and word prediction models. This makes it easy to isolate cases.
- Push models to Hugging Face’s Hub with one command.
- Enable saving models periodically during training.
- Preprocesses data is now saved in the Hugging Face’s Dataset format rather than in JSON format.
- Dictionary argument inputs for training and evaluating are no longer supported
- Removed adam_beta1, adam_beta2, adam_epsilon and max_grad_norm learning parameters.
- Replaced save_preprocessed_data, save_preprocessed_data_path with a single parameter called save_path. Likewise for load_preprocessed_data and load_preprocessed_data_path being replaced by load_path.
- Removed support for dictionary settings for the args parameter for training and evaluating.
- Removed the preprocessing_processes parameter for training and evaluating.
Introducing Version 2.4.0!
- We added the ability to enable half-precision training, which decreases train time and memory consumption. Just simply set the “fp16” training argument to True while using CUDA/ a GPU.
- We also set the character encoding format to utf-8 for HappyTextClassification and HappyQuestionAnswering. Before it would change based on your system.
- You can now use private models from Hugging Face’s Model Hub
Introducing Version 2.3.0!
- Text-to-text fine-tuning is now available!
Introducing Version 2.2.0!
- Text generation with training
- Word prediction training
- Saving/loading models
- Saving/loading preprocessed data
- You can now change the batch size when training and evaluating
- Dataclasses can now be used for all finetuning “arg” parameters
Introducing Version 2.1.0! You can now use any model type available on Hugging Face’s model distribution network for the implemented features. This includes BERT, ROBERTA, ALBERT XLNET and more.
You can also now perform token classification
Introducing Version 2.0.0!
We fully redesigned Happy Transformer from the ground up.
- Question answering training
- Multi label text classification training
- Single predictions for text classification
- Masked word prediction training
- Masked word prediction with multiple masks
Happy Transformer have been redesigned to promote scalability. Now it’s easier than ever to add new models and features, and we encourage you to create PRs to contribute to the project.