Networking, sockets, and UDP Streaming fun!


I’ve been tinkering with robotics lately and itching to put a camera on my Bluebot so I can watch it from my wristwatch as I drive it around. That got me looking for info on how to stream video from one device to another. Hi, I’m Cliff. You’re here because you probably want to have a device spew data out into the sky then receive it from another device using a UDP network connection. I had done UDP video streaming about 4-5 years ago as a prototype for my wife’s salon business and that time I got extremely lucky. She wanted a way to stream video from a webcam over the internet and I didn’t know where to start. I went low level and wrote a UDP video client/server in Java. I’ve since lost the code but it wasn’t good code. (It was a dirty hack where I was exceeding the practical size for each packet… and I don’t think the UDP stuff even worked. I think I converted to TCP at the last minute! There was a time limit for my experiment and I came in literally one second before time was up!)

Flash forward to today where I again have the need to stream video over the network. I found an old abandoned post on a Java forum where someone wrote a choppy Audio UDP client server. I don’t know why but I think this example, as crude as it is, looks pretty cool! The guy who posted the code wanted to know why the audio sounds so choppy. I looked at the code and found the problem in about 30 seconds without compiling, or even trying to run and listen to it! The problem is obvious, so I’m going to post the code here for you to read and ponder. I’ll try to work through a solution in my free time.

The server below attempts to record audio from the microphone and stream it via UDP to a client.

The Server:

public class MicPlayer {
 
    private static final String IP_TO_STREAM_TO   = "localhost" ;
    private static final int PORT_TO_STREAM_TO     = 8888 ;
 
    /** Creates a new instance of MicPlayer */
    public MicPlayer() {
 
    }
 
    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
    Mixer.Info minfo[] = AudioSystem.getMixerInfo() ;
    for( int i = 0 ; i < minfo.length ; i++ )
    {
     System.out.println( minfo[i] ) ;    
    }
 
 
    if (AudioSystem.isLineSupported(Port.Info.MICROPHONE)) {
    try {
 
 
      DataLine.Info dataLineInfo = new DataLine.Info( TargetDataLine.class , getAudioFormat() ) ;
      TargetDataLine targetDataLine = (TargetDataLine)AudioSystem.getLine( dataLineInfo  ) ;
      targetDataLine.open( getAudioFormat() );
      targetDataLine.start();
      byte tempBuffer[] = new byte[1000] ;
      int cnt = 0 ;
      while( true )
      {
      targetDataLine.read( tempBuffer , 0 , tempBuffer.length );
      sendThruUDP( tempBuffer ) ;
      }
 
    }
    catch(Exception e )
    {
    System.out.println(" not correct " ) ;
    System.exit(0) ;
    }
    }
 
 
 
    }
 
 
    public static AudioFormat getAudioFormat(){
    float sampleRate = 8000.0F;
    //8000,11025,16000,22050,44100
    int sampleSizeInBits = 16;
    //8,16
    int channels = 1;
    //1,2
    boolean signed = true;
    //true,false
    boolean bigEndian = false;
    //true,false
    return new AudioFormat( sampleRate, sampleSizeInBits, channels, signed, bigEndian );
    }
 
 
    public static void sendThruUDP( byte soundpacket[] )
    {
       try
       {
       DatagramSocket sock = new DatagramSocket() ; 
       sock.send( new DatagramPacket( soundpacket , soundpacket.length , InetAddress.getByName( IP_TO_STREAM_TO ) , PORT_TO_STREAM_TO ) ) ; 
       sock.close() ;
       }
       catch( Exception e )
       {
       e.printStackTrace() ;
       System.out.println(" Unable to send soundpacket using UDP " ) ;   
       }
 
    }
 
 
}

The client below attempts to receive the UDP packets and push them into the speakers for output:

The client:

public class RadioReceiver extends Thread {
 
    private static final String IP_TO_STREAM_TO   = "localhost" ;
    private static final int PORT_TO_STREAM_TO     = 8888 ;
 
    /** Creates a new instance of RadioReceiver */
    public RadioReceiver() {
    }
 
    public void run()
    {
        byte b[] = null ;
        while( true )
        {
           b = receiveThruUDP() ; 
           toSpeaker( b ) ;
        }        
    }
 
    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
 
    RadioReceiver r = new RadioReceiver() ;
    r.start() ;
 
    }
 
 
    public static byte[] receiveThruUDP()
    {
       try
       {
       DatagramSocket sock = new DatagramSocket(PORT_TO_STREAM_TO) ; 
       byte soundpacket[] = new byte[1000] ;
       DatagramPacket datagram = new DatagramPacket( soundpacket , soundpacket.length , InetAddress.getByName( IP_TO_STREAM_TO ) , PORT_TO_STREAM_TO ) ;
       sock.receive( datagram ) ; 
       sock.close() ;
       return datagram.getData() ; // soundpacket ;
       }
       catch( Exception e )
       {
       System.out.println(" Unable to send soundpacket using UDP " ) ;   
       return null ;
       } 
 
    }
 
 
     public static void toSpeaker( byte soundbytes[] )
     {
 
      try{  
      DataLine.Info dataLineInfo = new DataLine.Info( SourceDataLine.class , getAudioFormat() ) ;
      SourceDataLine sourceDataLine = (SourceDataLine)AudioSystem.getLine( dataLineInfo );
      sourceDataLine.open( getAudioFormat() ) ;
      sourceDataLine.start();
      int cnt = 0;
      sourceDataLine.write( soundbytes , 0, soundbytes.length );
      sourceDataLine.drain() ;
      sourceDataLine.close() ;
      }
      catch(Exception e )
      {
      System.out.println("not working in speakers " ) ;
      }
 
    }
 
 
    public static AudioFormat getAudioFormat()
    {
    float sampleRate = 8000.0F;
    //8000,11025,16000,22050,44100
    int sampleSizeInBits = 16;
    //8,16
    int channels = 1;
    //1,2
    boolean signed = true;
    //true,false
    boolean bigEndian = false;
    //true,false
    return new AudioFormat( sampleRate, sampleSizeInBits, channels, signed, bigEndian );
    }
 
 
}

Again, the complaint was that the audio sounds choppy and nobody on the abandoned thread could answer why. I wish there wasn’t a registration for posting because I so want to answer the obvious but I don’t want to fill out my email and register a user name and etc. Can you see where the problem is? I may be getting ahead of myself so I should actually see if the code compiles and works at all!

6 thoughts on “Networking, sockets, and UDP Streaming fun!

  1. UDP can work for streaming in a lossless environment, but in practical real-world networks you could receive duplicate datagrams, receive them out-of-order, or not at all. Converting to TCP was a smart move. While SMH about using UDP to stream audio, I skipped over code analysis. Hope you find the bug bro! (Though still, UDP? C’mon man, that’s the bug!).

  2. Geez. Ok. I will play. 1000 bytes is a little bigger than a datagram. So sending 1000 bytes results in one full and one partially full (zero padded). Skipping the padding nanifests as choppiness. Si?

  3. b = receiveThruUDP() ;
    toSpeaker( b ) ;

    Well, this is expected to be choppy because you are reading and posting packets sequentially in one thread. While you are reading, you can’t be writing, and vice versa. I would advise two threads and a blocking queue here (simplistically, an unbounded LinkedBlockingQueue, but if you are concerned enough to constrain possible memory allocation, then ArrayBlockingQueue is your choice.

    You may also want to pool DatagramPackets.

    1. That’s one way of thinking about it but you’re overthinking things. I posted the answer today, you can refer to part III of the series.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s