feat: add support for packing tokenized datasets

[kmehant]

What does this PR do?

Fixes #1848

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link
  to it if that's the case. Support packing for pretokenized datasets #1848
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines.
• Did you write any new necessary tests?

Who can review?

@qgallouedec

Anyone from the community!

[kmehant]

Anyone having bandwidth, requesting review thank you - @qgallouedec @lewtun @kashif @lvwerra or others from community.

Discussion can be seen here - #1848

[lewtun]

Thanks for the contribution @kmehant ! Overall it looks good to me - would you mind adding an integration test for this scenario?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[kmehant]

@lewtun Thank you for your review. I have addressed the review comments and as well added the test cases. Thank you.