Then why not start by looking at what audio libraries are available for linux, including finding out which ones are already installed on your system, and then choosing one that seems like it will handle the input formats you want to handle?