We present the design and implementation of a universal, single-bitstream library for accelerating matrix-vector multiplication using FPGAs. Our library is capable of handling multiple matrix encodings ranging from dense to multiple sparse formats. A key novelty in our approach is the introduction of an FPGA-optimized sparse matrix representation called Compressed Variable-Length Bit Vector (CVBV), which reduces the storage and bandwidth requirements up to 54% (on average 30%) compared to compressed sparse row (CSR) across 2000 inputs. Our hardware incorporates a runtimeprogrammable decoder that performs on-the-fly decoding of various formats such as Dense, COO, CSR, and ELL. The flexibility and scalability of our design is demonstrated across two FPGA platforms: (1) the BEE3 (Virtex-5 LX155T with 16GB of DRAM) and (2) ML605 (Virtex-6 LX240T with 2GB of DRAM). For dense matrices, our approach scales to large data sets with over 1 billion elements, and achieves robust performance independent of the matrix aspect ratio. For sparse matrices, our approach using a compressed representation reduces the overall bandwidth while also achieving comparable efficiency relative to state-of-the-art approaches.