Combine streams in one .NET Framework Stream object

Why Combine Multiple Streams?

In many .NET applications you end up working with a handful of data sources that are already wrapped in Stream objects – a file on disk, a network packet, a memory buffer, a database blob. When you need to expose a single, coherent view of all those pieces, the naive approach is to copy every source into a new MemoryStream and hand that back to the caller. That technique works fine for tiny files or a few small buffers, but it brings two major problems when the amount of data grows.

First, copying consumes extra memory. A MemoryStream holds all bytes in a contiguous array. If you concatenate several megabytes of data, the resulting array must allocate space for the sum of those sizes. Even with a generous memory limit, this can push the process into the paged memory zone, increasing swapping and slowing the whole application. Second, copying is a linear operation that touches each byte once. For large inputs this incurs a high CPU load and a noticeable delay before the caller can start processing. If the caller needs to read only the first few hundred bytes, the copy step is wasted effort.

The real question is: can we read or stream from a logical sequence of existing streams without physically merging them? The answer is yes, by creating a composite stream that walks through its constituent streams in order. Such a stream, which we’ll call MultiStream, behaves like a single Stream instance: you can Read, Seek, query Length, and even Dispose. Internally it keeps a list of the child streams and translates each read request into a series of reads on the appropriate child. This keeps memory usage to a minimum and removes the copy overhead.

When you decide that a composite stream is the right tool, you must consider its constraints. Because MultiStream simply forwards reads, it cannot modify the underlying data. The CanWrite property therefore returns false, and the Write method throws NotSupportedException. That limitation is intentional; the stream is read‑only by design. If you need to build a writable composite, you would have to create a custom write strategy that writes to the right child based on the current position – a considerably more complex problem.

Another subtlety is the handling of position and length. The composite stream must report its length as the sum of the lengths of all child streams. Seeking to an arbitrary offset means that the implementation must determine which child stream contains that position and adjust the internal cursor accordingly. While these calculations are inexpensive, they require careful bookkeeping to avoid off‑by‑one errors. The following sections walk through a practical implementation of MultiStream, then show how to use it in a real codebase.

This pattern is common in scenarios such as building a virtual file system, concatenating multiple log files into a single view, or delivering a chunked HTTP response that combines a static header stream, a body stream, and a trailer stream. By avoiding data copying you keep your application lean, responsive, and scalable to large datasets.

Building a MultiStream Class

The core of a composite stream is the logic that maps a global read request onto the underlying streams. In .NET you inherit from System.IO.Stream and override the abstract members: Read, Write, Seek, Length, Position, and a few others. The simplest implementation of the write members just throws NotSupportedException because the composite is read‑only.

Below is a streamlined implementation that focuses on the key behaviors. The full source, including helper methods and unit tests, can be downloaded from multistreamcs.zip. The class holds a list of child streams and a precomputed array of cumulative offsets. This array speeds up the Seek operation by allowing a binary search instead of walking the list each time.

The Read method is the heart of the composite. It first checks whether the requested position lies beyond the end of the stream; if so it returns zero to signal EOF. It then loops until either the requested byte count is satisfied or the global end is reached. For each iteration the method determines which child stream contains the current global position via FindStreamIndex, seeks that child to the correct local offset, and reads up to the requested amount or the child’s remaining bytes. After each read, the global position and buffer pointers are advanced accordingly. The loop continues until the buffer is full or no more data is available.

Seeking is straightforward because the cumulative lengths array gives you a quick reference. The Seek method calculates the new position based on the origin, clamps it between zero and Length, and updates the internal _position. Subsequent reads will automatically map to the right child.

Because MultiStream does not own the child streams (they are passed in), the Dispose implementation simply forwards the dispose call to each child. This is a design choice: if you want to keep the child streams open after the composite is disposed, remove that loop and let the caller manage disposal.

The implementation deliberately stays lightweight. It does not expose methods for adding or removing streams after construction, but you could extend it by adding an AddStream method that updates the cumulative offsets. That extension would be useful for scenarios where streams are discovered lazily.

With the class in place, the next step is to see how it integrates into everyday coding tasks, such as concatenating a string stream with a file stream and reading the combined content. The following example demonstrates that.

Practical Use Cases

To illustrate the power of MultiStream, consider a situation where you need to serve a composite resource over HTTP: a small header block, the body of a file, and a trailer. You could create separate streams for each part and hand them to MultiStream. The caller can then treat the entire payload as a single stream, making it trivial to pipe to an HttpResponse body or to a file write operation.

Below is a concise test harness that demonstrates the composite stream in action. It builds a MultiStream from a string buffer and a real file, then reads the first 1,024 bytes. The test validates that the returned data equals the expected concatenation.

In this example the string part is small and resides entirely in memory, whereas the file part can be arbitrarily large. The Read call will read from the string stream until it reaches its end, then seamlessly continue with the file stream. No copying occurs; the data flows directly from each source to the buffer supplied by the caller.

A slightly more elaborate scenario involves reading from multiple files in a directory. You could create a list of FileStream objects and pass them to MultiStream. The composite will expose the concatenated size of all files. Iterating over the stream will produce a single continuous byte sequence that you can process in a single pass, which is especially handy when generating checksums or streaming data over a network.

Because MultiStream implements Seek, you can jump to any position within the virtual concatenated stream. This allows random access to parts of the composite without having to reopen or reposition each child stream separately. The only caveat is that the seeking algorithm assumes that each child stream supports seeking; if any child stream is non‑seekable (for instance a network stream), you must redesign the composite or provide a custom seek strategy.

When working in a multi‑threaded environment, remember that Stream objects are not thread‑safe by default. If you need concurrent reads from a MultiStream, you should guard access with a lock or otherwise ensure that only one thread uses the stream at a time.

Overall, the MultiStream class gives you a clean abstraction for treating a sequence of independent streams as one. It keeps memory usage minimal, eliminates copy overhead, and preserves the familiar Stream interface, making it an ideal building block for high‑performance I/O scenarios.

If you’d like to explore further, download the source archive linked above. The zip contains not only the MultiStream.cs file but also a small console application that demonstrates several use cases and unit tests that verify correctness under a variety of conditions.

For additional resources on stream manipulation in .NET, you can sign up for free technical newsletters at info@clevercomponents.com. The authors at Clever Components are known for providing high‑quality tools and consulting for developers working with Delphi, C++, and Interbase, and they welcome feedback on this stream implementation.