Instruction-tuning Support

[lllAlexanderlll]

What does this PR do?

This PR adds support for instruction tuning, by

1. Introducing a new entry point data apply_chat_template --config_file_path config_files/training/config_lorem_ipsum_sft.yaml which takes a instruction-dataset and converts the structured conversations into a single prompt by applying the chat template given as jinja2 template string within the config. Here, we also include indicator tokens to mark what the system utterances are.
2. In modalities training entry point you can now wrap the collate function by a "LossMaskingCollateFn", which first executes the wrapped collate function and then applies loss masking on each target as specified in the config. This allows to only include tokens that are part of the assistant answer into the loss, so that the model learns to act as helpful assistant.
3. Modifiy the PackedMemMapDatasetContinuous to allow not to re-use the last target token, as this is not wanted in instruction-tuning where we apply truncation and packing.

General Changes

• New entry point data apply_chat_template --config_file_path config_files/training/config_lorem_ipsum_sft.yaml to convert structured JSONL into JSONL with a the new attribute "chat", i.e. the prompt were the chat template was applied
• A wrapper for collate functions to include tokens which appear between indicator tokens
• A new parameter for the PackedMemMapDatasetContinuous to allow not to re-use the last target token

Breaking Changes

• None, as the default value for PackedMemMapDatasetContinuous.reuse_last_target is True

Checklist before submitting final PR

• My PR is minimal and addresses one issue in isolation
• I have merged the latest version of the target branch into this feature branch
• I have reviewed my own code w.r.t. correct implementation, missing type hints, proper documentation, etc.
• I have run a sample config for model training
• I have checked that all tests run through (python tests/tests.py)

[le1nux]

so basically we represent the special token by at least on of the original tokens in the vocabulary?

For example, assuming "spec_tok_1" is supposed to be a special token, it gets tokenized to e.g., ["spec", "_", "tok", "_", "1"] ? Since it has been added to the Trie, we therefore still know that it is a special token but during training and inference the special token would still be represented by a 5 token sequence and therefore the embedding matrix can remain unchanged, right?

What happens if the special token cannot be represented?

[lllAlexanderlll]

Not quite.
Given spec_tok_1 as a special token, which is part of the trie and given the text to tokenize being Hi there spec_tok_1!:
First, the split method of the trie is used to extract all special tokens. In our case, the split would look like this: ["Hi there", "spec_tok_1", "!"], then all strings that are not part of the trie, i.e., the normal tokens, are tokenized leading to e.g. ["Hi", "there", "spec_tok_1", "!"]