[core] Replace QuantLlamaMLP with QuantFusedMLP

[younesbelkada]

In the context of huggingface/transformers#27411 I wanted to create a more generic class for fused MLP layers. I kept the previous QuantLlamaMLP for backward compatiblity

cc @casper-hansen

This PR also fixed another issue where users face a strange issue whenever they try to perform multi-turn generation.

[casper-hansen]

Nice one @younesbelkada! 👍

What do you think about setting a default activation=F.silu? Then we can use QuantFusedMLP internally in AutoAWQ as well when migrating from QuantLlamaMLP.

[younesbelkada]

Sounds great! Happy to address these changes

[younesbelkada]

Done ! LMK what do you think

[casper-hansen]

I am accepting this PR for now to unblock further work on the integration with transformers. However, I do intend to run more tests to confirm the last patch for multiple generations does not cause bugs with inference in AutoAWQ.

I might also consider implementing ‘is_transformers’ and default it to True to make sure transformers has explicitly different behavior which is evidently needed in some cases.