InputStream audio mixing (MODE_STREAM)

I'm making a drum sequencer in Android...

I'm writing to an AudioTrack in MODE_STREAM, so that I can achieve synchronized audio playback with all InputStreams (availible via a list of 'active' InputStreams, activeStreams in the code below)

The audio is always: PCM (WAV), 16bit Stereo 44100 Hz.

Obviously, I can't composite audio in real time on the UI thread, so I'm using an AsyncTask to queue up all the audio buffering.

I got buffered playback working, but when it comes to merging the buffers of two (or more) InputStream's, the internet seems to be in some kind of debate of what to do next. "Convert the byte[] to short[]!", "No, do the bit mixing on-the-fly!", "But if you don't use shorts the byte Endianness is ignored!", "It gets ignored anyway!" - I don't even know any more.

How do I mix the buffer of two or more InputStreams? I don't understand why my current implementation is failing

I've tried like, 4 different StackOverflow solutions to convert the byte[] to short[] so I can add the samples together, but the conversion always instantly crashes Java with some cryptic error message that I can't get my head around. So now I give up. Here's my code implementing one such StackOverflow solution...

protected Long doInBackground(Object ... Object) {

  int bytesWritten = 0;
  InputStream inputStream;
  int si = 0, i = 0;

  //The combined buffers. The 'composition'
  short[] cBuffer = new short[Synth.AUDIO_BUFFER_SIZE];

  //The 'current buffer', the segment of inputStream audio.
  byte[] bBuffer = new byte[Synth.AUDIO_BUFFER_SIZE];

  //The 'current buffer', converted to short?
  short[] sBuffer = new short[Synth.AUDIO_BUFFER_SIZE];

  int curStreamNum;
  int numStreams = activeStreams.size();
  short mix;

  //Start with an empty 'composition'
  cBuffer = new short[Synth.AUDIO_BUFFER_SIZE];

  boolean bufferEmpty = false;
  try {
    while(true) { // keep going forever, until stopped or paused.
      for(curStreamNum = 0;curStreamNum < numStreams;curStreamNum++){
        inputStream = activeStreams.get(curStreamNum);
        i = inputStream.read(bBuffer);
        bufferEmpty = i<=-1;
        if(bufferEmpty){
          //Input stream buffer was empty. It's out of audio. Close and remove the stream.
          inputStream.close();
          activeStreams.remove(curStreamNum);
          curStreamNum--; numStreams--; continue; // hard continue.
        }else{
          //Take the now-read buffer, and convert to shorts.
          ByteBuffer.wrap(bBuffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(sBuffer);

          //Take the short buffer, merge into composition buffer.
          //TODO: Optimize by making the 'first layer' of the composition the first buffer, on its own.
          for(si=0;si<Synth.AUDIO_BUFFER_SIZE;si++){
            mix = (short) (sBuffer[si] + cBuffer[si]);
            //This part is probably completely wrong too. I'm not up to here yet to evaluate whats needed...
            if(mix >= 32767){
              mix = 32767;
            }else if (mix <= -32768){
              mix = -32768;
            }
            cBuffer[si] = mix;
          }
        }
      }
      track.write(sBuffer, 0, i);

      //It's always full; full buffer of silence, or of composited audio.
      totalBytesWritten += Synth.AUDIO_BUFFER_SIZE;

      //.. queueNewInputStreams ..
      publishProgress(totalBytesWritten);
      if (isCancelled()) break;
    }
  } catch (IOException e) {e.printStackTrace();}
  return Long.valueOf(totalBytesWritten);
}

I'm currently getting a BufferUnderflowException on this line: ByteBuffer.wrap(bBuffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(sBuffer);.

How is it possible to have buffer underrun? I'm only converting a byte[] to a short[].

Please help!

I've posted my whole function in the hopes that this more complete code sample and fairly adaptable usage can help other people out there.

(P.S. the byte[] to short[] conversion is followed by some flimsy hard clipping which I'm not even up to debugging yet, but advice there would also be appreciated)