2023年9月19日 · In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images. …

The overall network architecture of Qwen-VL consists of three components and the details of model parameters are shown in Table 1: Large Language Model: Qwen-VL adopts a large …

Published as a conference paper at ICLR 2024 3.1 LOSS OF HIGH FREQUENCY INFORMATION - "NTK-AWARE" INTERPOLATION If we look at rotary position embeddings …

2024年9月12日 · Our experiments demonstrate AIR's effectiveness across state-of-the-art LLMs, achieving an attack success rate (ASR) exceeding $\textbf{90}$% on most models, including …

2024年9月26日 · Additionally, we evaluate three categories of LLMs against this benchmark: 1) proprietary LLM services such as GPT-4; 2) public LLMs like QWen; and 3) specialized …

Welcome to the OpenReview homepage for ICLR 2024

tuning phase. To lift the OCR accuracy and support other languages, e.g., Chinese, Qwen-VL Bai et al. (2023b) unfreezes its image encoder (a CLIP-G) and uses lots of OCR data in its stage …

2024年1月16日 · The aligned Large Language Models (LLMs) are powerful language understanding and decision-making tools that are created through extensive alignment with …

Building upon this formulation, the experts in both Qwen (Team, 2024) and Mixtral (Jiang et al., 2024) adopt the structure of LLaMA (Touvron et al., 2023). Specifically, the feed-forward …

We propsoe MARVEL to adaptively allocate the parameter budget among weight matrices in correspondence to their importance.