Java NIO: Buffers

If you've worked with binary file formats, for example in the fields of data recovery or file compatibility, you'll be familiar with the idea of processing "blocks" of data from a file or stream, or processing data as groups of bytes that function together as other types of data such as 4-byte words, Unicode strings etc. For example, you might need to read data in from a file that consists of a "4 byte header", followed by a "4-byte offset" to something, followed by a "2-byte length" etc. Or in a database file, you might need to treat the file as fixed-size "pages" where you read in a page of data from a particular offset, modify it in memory, then write it back to the file.

In the standard java.io package, a file or stream is generally presented to the programmer as a series of bytes or byte array(s). The DataInput and DataOutput interfaces can help out, but they still only work on streams, and crucially don't allow byte order to be specified. Operations such as "read a little-endian 4-byte integer from offset 10" are clumsy in this stream-based model.

This is where the NIO Buffer class and its subclasses come in. A commonly used buffer class is ByteBuffer. This and its cousins:

• allow a byte array (or block of memory/file) to be managed as a buffer, with methods to deal with the current read/write position, maximum position, byte order etc;
• provide convenience methods such as putInt(), putLong() etc for accessing specific sizes of data and at a particular offset;
• are optimized by the JVM (at least in Sun's implementation) so that, for example, a Buffer.get() method can literally be translated into a move instruction at the machine-code level;
• can provide a mapped view of a file so that, for example, calling putShort() with a particular offset can literally write to that offset in the file.

Additionally, a type of buffer called a direct buffer provides access to a fixed area of memory allocated outside the Java heap.

At present, buffer sizes are limited to 2GB (the maximum positive number that can be represented in an int. An updated planned for Java 7 will allow large buffers (with the size and indexes held as a long).

On the next page, we'll start by having a look at the layout of a NIO buffer. This will help us understand the various methods that can be called on a buffer. We'll then look at the various types of buffer offered as part of the java.nio package, and crucially the ByteBuffer, probably the most common and generic NIO buffer class.

---