Improving software performance with buffers

The efficient use of data structures and algorithms is central to building software that is fast, scalable, and cheap to run. Buffers are one of the clearest examples of this, because the idea is simple and the payoff is immediate.

What buffers are

A buffer is a temporary holding area for data in transit between two places that work at different speeds. That description sounds too general to be useful, but the speed mismatch it papers over is often enormous — memory and disk differ by several orders of magnitude — and closing it is one of the cheapest performance wins available.

Buffers turn up across a wide range of applications: networking, file systems, audio and video processing, and anywhere else data is moved between components. They are especially valuable when the rate of data production differs from the rate of consumption, which is exactly the situation when reading from or writing to disk.

Why they help

Every unbuffered write is a system call, and a system call is expensive relative to a memory copy: the CPU switches into the kernel, does the work, and switches back. Doing that once per line means paying the transition cost once per line.

A buffer collects writes in memory and hands them over in batches. The bytes written are identical; the number of crossings into the kernel is not, and that is the entire saving.

There is a second benefit worth noting. Because a buffer decouples the producer from the consumer, software can continue processing data while a transfer is in progress rather than blocking until the whole transfer completes. Reducing I/O operations makes things faster; decoupling the two sides makes them independent.

How Go implements them

In Go, buffered I/O lives in the bufio package, which provides buffered operations for files, input and output streams, and any other reader or writer. Using it reduces the number of system calls a program makes without changing how the code reads.

Internally, bufio uses a fixed-size byte slice as the buffer. For a writer, data accumulates in the slice as it is written. Once the slice is full, its entire contents are flushed to the underlying destination in a single I/O operation and the buffer resets — one large write instead of many small ones. For a reader, the mechanism runs in reverse: a large chunk is read from the source into the slice, and subsequent reads are served from memory until the buffer empties and needs refilling. The size of that slice sets the batch size, which in turn sets how many system calls you end up making.

The bookkeeping is done with two pointers, r and w. The r pointer marks the next byte to be read from the buffer; the w pointer marks the next position to be written. The difference between them is the number of bytes currently held. When the buffer fills, w wraps back to the beginning so new data can be written from the start once the existing contents have been consumed.

Measuring it

To see the difference concretely, here is a program that writes the numbers 0 to 99,999 to a file called test.txt, once directly and once through a bufio.Writer, timing each with time.Since:

package main
 
import (
	"bufio"
	"fmt"
	"os"
	"time"
)
 
func main() {
	file, err := os.Create("test.txt")
	if err != nil {
		panic(err)
	}
	defer file.Close()
 
	start := time.Now()
	for i := 0; i < 100000; i++ {
		fmt.Fprintln(file, i)
	}
	fmt.Printf("Without buffer: %s\n", time.Since(start))
 
	start = time.Now()
	w := bufio.NewWriter(file)
	for i := 0; i < 100000; i++ {
		fmt.Fprintln(w, i)
	}
	w.Flush()
	fmt.Printf("With buffer:    %s\n", time.Since(start))
}

Running it gives:

Without buffer: 9.357391ms
With buffer:    3.488088ms

Roughly 2.7× faster for the same 100,000 lines — more than halving the time taken. The work done is identical; the number of trips into the kernel is not, because the buffer groups many small writes into far fewer large ones.

Treat the exact numbers as indicative rather than precise — they depend on the filesystem, the page cache, and whatever else the machine is doing. The ratio is the durable part, and it grows as the writes get smaller and more frequent.

The mistake everyone makes once

file, _ := os.Create("out.txt")
defer file.Close()
 
w := bufio.NewWriter(file)
defer w.Flush()

Where else this applies

The pattern generalises well past file I/O, because the underlying shape — a per-operation fixed cost that batching amortises — turns up constantly:

• Network writes, where the fixed cost is a packet rather than a syscall.
• Database inserts, where one statement with a thousand rows beats a thousand statements.
• Log shipping, where batching is the difference between a viable pipeline and a self-inflicted denial of service.

The trade is latency against throughput. Buffered data is data that has not arrived yet, so a large buffer on a low-traffic connection means messages sit around waiting for company. For a log file that is free. For an interactive protocol it is a bug, which is why terminals line-buffer instead.

Getting this right is mostly a matter of asking what the fixed cost per operation is, and whether anything is waiting on the result.

Conclusion

Buffers are a good example of how a small amount of attention to data structures translates directly into performance. By reducing the number of I/O operations a program performs, they deliver substantial gains for very little code — in this case a single wrapper type and one call to Flush.

The broader lesson is that the win did not come from doing less work. It came from doing the same work in fewer, larger pieces, and recognising where that restructuring is available is a large part of what performance work actually consists of.