No AI summary available for this article.
Why It Matters
Model merging combines multiple task-specific fine-tuned LLMs into a single multi-task model without additional training.
Provenance
Discovered via ArXiv and published by ArXiv.
Key Claims
Original description
Model merging combines multiple task-specific fine-tuned LLMs into a single multi-task model without additional training. However, merged models are known to suffer from representation bias: systematic drift between the merged model's hidden states and those of each individual source model. Prior work (Yang et al., 2024a) study and mitigate this bias for encoder-based vision models using a lightweight correction module trained with L1 loss. However, such bias is not studied for decoder models due to their autoregressive nature. We analyze the problem of representation bias in decoder models, a...
Discovered via ArXiv
Research papers and preprints from arXiv.
Publisher: arxiv.org
ID: http://arxiv.org/abs/2608.28547v1 · Indexed 15 days ago