A stream is an abstract interface implemented by various objects in\nNode. For example a request to an HTTP\nserver is a stream, as is\n[stdout][]. Streams are readable, writable, or both. All streams are\ninstances of [EventEmitter][]\n\n
\nYou can load the Stream base classes by doing require('stream').\nThere are base classes provided for [Readable][] streams, [Writable][]\nstreams, [Duplex][] streams, and [Transform][] streams.\n\n
This document is split up into 3 sections. The first explains the\nparts of the API that you need to be aware of to use streams in your\nprograms. If you never implement a streaming API yourself, you can\nstop there.\n\n
\nThe second section explains the parts of the API that you need to use\nif you implement your own custom streams yourself. The API is\ndesigned to make this easy for you to do.\n\n
\nThe third section goes into more depth about how streams work,\nincluding some of the internal mechanisms and functions that you\nshould probably not modify unless you definitely know what you are\ndoing.\n\n\n
\n", "classes": [ { "textRaw": "Class: stream.Readable", "type": "class", "name": "stream.Readable", "desc": "The Readable stream interface is the abstraction for a source of\ndata that you are reading from. In other words, data comes out of a\nReadable stream.\n\n
\nA Readable stream will not start emitting data until you indicate that\nyou are ready to receive it.\n\n
\nReadable streams have two "modes": a flowing mode and a non-flowing\nmode. When in flowing mode, data is read from the underlying system\nand provided to your program as fast as possible. In non-flowing\nmode, you must explicitly call stream.read() to get chunks of data\nout.\n\n
Examples of readable streams include:\n\n
\nWhen a chunk of data can be read from the stream, it will emit a\n'readable' event.\n\n
In some cases, listening for a 'readable' event will cause some data\nto be read into the internal buffer from the underlying system, if it\nhadn't already.\n\n
var readable = getReadableStreamSomehow();\nreadable.on('readable', function() {\n // there is some data to read now\n})Once the internal buffer is drained, a readable event will fire\nagain when more data is available.\n\n
If you attach a data event listener, then it will switch the stream\ninto flowing mode, and data will be passed to your handler as soon as\nit is available.\n\n
If you just want to get all the data out of the stream as fast as\npossible, this is the best way to do so.\n\n
\nvar readable = getReadableStreamSomehow();\nreadable.on('data', function(chunk) {\n console.log('got %d bytes of data', chunk.length);\n})This event fires when there will be no more data to read.\n\n
\nNote that the end event will not fire unless the data is\ncompletely consumed. This can be done by switching into flowing mode,\nor by calling read() repeatedly until you get to the end.\n\n
var readable = getReadableStreamSomehow();\nreadable.on('data', function(chunk) {\n console.log('got %d bytes of data', chunk.length);\n})\nreadable.on('end', function() {\n console.log('there will be no more data.');\n});Emitted when the underlying resource (for example, the backing file\ndescriptor) has been closed. Not all streams will emit this.\n\n
\n", "params": [] }, { "textRaw": "Event: 'error'", "type": "event", "name": "error", "params": [], "desc": "Emitted if there was an error receiving data.\n\n
\n" } ], "methods": [ { "textRaw": "readable.read([size])", "type": "method", "name": "read", "signatures": [ { "return": { "textRaw": "Return {String | Buffer | null} ", "name": "return", "type": "String | Buffer | null" }, "params": [ { "textRaw": "`size` {Number} Optional argument to specify how much data to read. ", "name": "size", "type": "Number", "desc": "Optional argument to specify how much data to read.", "optional": true } ] }, { "params": [ { "name": "size", "optional": true } ] } ], "desc": "The read() method pulls some data out of the internal buffer and\nreturns it. If there is no data available, then it will return\nnull.\n\n
If you pass in a size argument, then it will return that many\nbytes. If size bytes are not available, then it will return null.\n\n
If you do not specify a size argument, then it will return all the\ndata in the internal buffer.\n\n
This method should only be called in non-flowing mode. In\nflowing-mode, this method is called automatically until the internal\nbuffer is drained.\n\n
\nvar readable = getReadableStreamSomehow();\nreadable.on('readable', function() {\n var chunk;\n while (null !== (chunk = readable.read())) {\n console.log('got %d bytes of data', chunk.length);\n }\n});Call this function to cause the stream to return strings of the\nspecified encoding instead of Buffer objects. For example, if you do\nreadable.setEncoding('utf8'), then the output data will be\ninterpreted as UTF-8 data, and returned as strings. If you do\nreadable.setEncoding('hex'), then the data will be encoded in\nhexadecimal string format.\n\n
This properly handles multi-byte characters that would otherwise be\npotentially mangled if you simply pulled the Buffers directly and\ncalled buf.toString(encoding) on them. If you want to read the data\nas strings, always use this method.\n\n
var readable = getReadableStreamSomehow();\nreadable.setEncoding('utf8');\nreadable.on('data', function(chunk) {\n assert.equal(typeof chunk, 'string');\n console.log('got %d characters of string data', chunk.length);\n})This method will cause the readable stream to resume emitting data\nevents.\n\n
This method will switch the stream into flowing-mode. If you do not\nwant to consume the data from a stream, but you do want to get to\nits end event, you can call readable.resume() to open the flow of\ndata.\n\n
var readable = getReadableStreamSomehow();\nreadable.resume();\nreadable.on('end', function(chunk) {\n console.log('got to the end, but did not read anything');\n})This method will cause a stream in flowing-mode to stop emitting\ndata events. Any data that becomes available will remain in the\ninternal buffer.\n\n
This method is only relevant in flowing mode. When called on a\nnon-flowing stream, it will switch into flowing mode, but remain\npaused.\n\n
\nvar readable = getReadableStreamSomehow();\nreadable.on('data', function(chunk) {\n console.log('got %d bytes of data', chunk.length);\n readable.pause();\n console.log('there will be no more data for 1 second');\n setTimeout(function() {\n console.log('now data will start flowing again');\n readable.resume();\n }, 1000);\n})This method pulls all the data out of a readable stream, and writes it\nto the supplied destination, automatically managing the flow so that\nthe destination is not overwhelmed by a fast readable stream.\n\n
\nMultiple destinations can be piped to safely.\n\n
\nvar readable = getReadableStreamSomehow();\nvar writable = fs.createWriteStream('file.txt');\n// All the data from readable goes into 'file.txt'\nreadable.pipe(writable);This function returns the destination stream, so you can set up pipe\nchains like so:\n\n
\nvar r = fs.createReadStream('file.txt');\nvar z = zlib.createGzip();\nvar w = fs.createWriteStream('file.txt.gz');\nr.pipe(z).pipe(w);For example, emulating the Unix cat command:\n\n
process.stdin.pipe(process.stdout);By default [end()][] is called on the destination when the source stream\nemits end, so that destination is no longer writable. Pass { end:\nfalse } as options to keep the destination stream open.\n\n
This keeps writer open so that "Goodbye" can be written at the\nend.\n\n
reader.pipe(writer, { end: false });\nreader.on('end', function() {\n writer.end('Goodbye\\n');\n});Note that process.stderr and process.stdout are never closed until\nthe process exits, regardless of the specified options.\n\n
This method will remove the hooks set up for a previous pipe() call.\n\n
If the destination is not specified, then all pipes are removed.\n\n
\nIf the destination is specified, but no pipe is set up for it, then\nthis is a no-op.\n\n
\nvar readable = getReadableStreamSomehow();\nvar writable = fs.createWriteStream('file.txt');\n// All the data from readable goes into 'file.txt',\n// but only for the first second\nreadable.pipe(writable);\nsetTimeout(function() {\n console.log('stop writing to file.txt');\n readable.unpipe(writable);\n console.log('manually close the file stream');\n writable.end();\n}, 1000);This is useful in certain cases where a stream is being consumed by a\nparser, which needs to "un-consume" some data that it has\noptimistically pulled out of the source, so that the stream can be\npassed on to some other party.\n\n
\nIf you find that you must often call stream.unshift(chunk) in your\nprograms, consider implementing a [Transform][] stream instead. (See API\nfor Stream Implementors, below.)\n\n
// Pull off a header delimited by \\n\\n\n// use unshift() if we get too much\n// Call the callback with (error, header, stream)\nvar StringDecoder = require('string_decoder').StringDecoder;\nfunction parseHeader(stream, callback) {\n stream.on('error', callback);\n stream.on('readable', onReadable);\n var decoder = new StringDecoder('utf8');\n var header = '';\n function onReadable() {\n var chunk;\n while (null !== (chunk = stream.read())) {\n var str = decoder.write(chunk);\n if (str.match(/\\n\\n/)) {\n // found the header boundary\n var split = str.split(/\\n\\n/);\n header += split.shift();\n var remaining = split.join('\\n\\n');\n var buf = new Buffer(remaining, 'utf8');\n if (buf.length)\n stream.unshift(buf);\n stream.removeListener('error', callback);\n stream.removeListener('readable', onReadable);\n // now the body of the message can be read from the stream.\n callback(null, header, stream);\n } else {\n // still reading the header.\n header += str;\n }\n }\n }\n}Versions of Node prior to v0.10 had streams that did not implement the\nentire Streams API as it is today. (See "Compatibility" below for\nmore information.)\n\n
\nIf you are using an older Node library that emits 'data' events and\nhas a pause() method that is advisory only, then you can use the\nwrap() method to create a [Readable][] stream that uses the old stream\nas its data source.\n\n
You will very rarely ever need to call this function, but it exists\nas a convenience for interacting with old Node programs and libraries.\n\n
\nFor example:\n\n
\nvar OldReader = require('./old-api-module.js').OldReader;\nvar oreader = new OldReader;\nvar Readable = require('stream').Readable;\nvar myReader = new Readable().wrap(oreader);\n\nmyReader.on('readable', function() {\n myReader.read(); // etc.\n});The Writable stream interface is an abstraction for a destination\nthat you are writing data to.\n\n
\nExamples of writable streams include:\n\n
\nThis method writes some data to the underlying system, and calls the\nsupplied callback once the data has been fully handled.\n\n
\nThe return value indicates if you should continue writing right now.\nIf the data had to be buffered internally, then it will return\nfalse. Otherwise, it will return true.\n\n
This return value is strictly advisory. You MAY continue to write,\neven if it returns false. However, writes will be buffered in\nmemory, so it is best not to do this excessively. Instead, wait for\nthe drain event before writing more data.\n\n
Call this method when no more data will be written to the stream. If\nsupplied, the callback is attached as a listener on the finish event.\n\n
Calling [write()][] after calling [end()][] will raise an error.\n\n
// write 'hello, ' and then end with 'world!'\nvar file = fs.createWriteStream('example.txt');\nfile.write('hello, ');\nfile.end('world!');\n// writing more now is not allowed!If a [writable.write(chunk)][] call returns false, then the drain\nevent will indicate when it is appropriate to begin writing more data\nto the stream.\n\n
// Write the data to the supplied writable stream 1MM times.\n// Be attentive to back-pressure.\nfunction writeOneMillionTimes(writer, data, encoding, callback) {\n var i = 1000000;\n write();\n function write() {\n var ok = true;\n do {\n i -= 1;\n if (i === 0) {\n // last time!\n writer.write(data, encoding, callback);\n } else {\n // see if we should continue, or wait\n // don't pass the callback, because we're not done yet.\n ok = writer.write(data, encoding);\n }\n } while (i > 0 && ok);\n if (i > 0) {\n // had to stop early!\n // write some more once it drains\n writer.once('drain', write);\n }\n }\n}When the [end()][] method has been called, and all data has been flushed\nto the underlying system, this event is emitted.\n\n
var writer = getWritableStreamSomehow();\nfor (var i = 0; i < 100; i ++) {\n writer.write('hello, #' + i + '!\\n');\n}\nwriter.end('this is the end\\n');\nwriter.on('finish', function() {\n console.error('all writes are now complete.');\n});This is emitted whenever the pipe() method is called on a readable\nstream, adding this writable to its set of destinations.\n\n
var writer = getWritableStreamSomehow();\nvar reader = getReadableStreamSomehow();\nwriter.on('pipe', function(src) {\n console.error('something is piping into the writer');\n assert.equal(src, reader);\n});\nreader.pipe(writer);This is emitted whenever the [unpipe()][] method is called on a\nreadable stream, removing this writable from its set of destinations.\n\n
var writer = getWritableStreamSomehow();\nvar reader = getReadableStreamSomehow();\nwriter.on('unpipe', function(src) {\n console.error('something has stopped piping into the writer');\n assert.equal(src, reader);\n});\nreader.pipe(writer);\nreader.unpipe(writer);Emitted if there was an error when writing or piping data.\n\n
\n" } ] }, { "textRaw": "Class: stream.Duplex", "type": "class", "name": "stream.Duplex", "desc": "Duplex streams are streams that implement both the [Readable][] and\n[Writable][] interfaces. See above for usage.\n\n
\nExamples of Duplex streams include:\n\n
\nTransform streams are [Duplex][] streams where the output is in some way\ncomputed from the input. They implement both the [Readable][] and\n[Writable][] interfaces. See above for usage.\n\n
\nExamples of Transform streams include:\n\n
\nStreams can be either [Readable][], [Writable][], or both ([Duplex][]).\n\n
\nAll streams are EventEmitters, but they also have other custom methods\nand properties depending on whether they are Readable, Writable, or\nDuplex.\n\n
\nIf a stream is both Readable and Writable, then it implements all of\nthe methods and events below. So, a [Duplex][] or [Transform][] stream is\nfully described by this API, though their implementation may be\nsomewhat different.\n\n
\nIt is not necessary to implement Stream interfaces in order to consume\nstreams in your programs. If you are implementing streaming\ninterfaces in your own program, please also refer to\n[API for Stream Implementors][] below.\n\n
\nAlmost all Node programs, no matter how simple, use Streams in some\nway. Here is an example of using Streams in a Node program:\n\n
\nvar http = require('http');\n\nvar server = http.createServer(function (req, res) {\n // req is an http.IncomingMessage, which is a Readable Stream\n // res is an http.ServerResponse, which is a Writable Stream\n\n var body = '';\n // we want to get the data as utf8 strings\n // If you don't set an encoding, then you'll get Buffer objects\n req.setEncoding('utf8');\n\n // Readable streams emit 'data' events once a listener is added\n req.on('data', function (chunk) {\n body += chunk;\n })\n\n // the end event tells you that you have entire body\n req.on('end', function () {\n try {\n var data = JSON.parse(body);\n } catch (er) {\n // uh oh! bad json!\n res.statusCode = 400;\n return res.end('error: ' + er.message);\n }\n\n // write back something interesting to the user:\n res.write(typeof data);\n res.end();\n })\n})\n\nserver.listen(1337);\n\n// $ curl localhost:1337 -d '{}'\n// object\n// $ curl localhost:1337 -d '"foo"'\n// string\n// $ curl localhost:1337 -d 'not json'\n// error: Unexpected token oThe Readable stream interface is the abstraction for a source of\ndata that you are reading from. In other words, data comes out of a\nReadable stream.\n\n
\nA Readable stream will not start emitting data until you indicate that\nyou are ready to receive it.\n\n
\nReadable streams have two "modes": a flowing mode and a non-flowing\nmode. When in flowing mode, data is read from the underlying system\nand provided to your program as fast as possible. In non-flowing\nmode, you must explicitly call stream.read() to get chunks of data\nout.\n\n
Examples of readable streams include:\n\n
\nWhen a chunk of data can be read from the stream, it will emit a\n'readable' event.\n\n
In some cases, listening for a 'readable' event will cause some data\nto be read into the internal buffer from the underlying system, if it\nhadn't already.\n\n
var readable = getReadableStreamSomehow();\nreadable.on('readable', function() {\n // there is some data to read now\n})Once the internal buffer is drained, a readable event will fire\nagain when more data is available.\n\n
If you attach a data event listener, then it will switch the stream\ninto flowing mode, and data will be passed to your handler as soon as\nit is available.\n\n
If you just want to get all the data out of the stream as fast as\npossible, this is the best way to do so.\n\n
\nvar readable = getReadableStreamSomehow();\nreadable.on('data', function(chunk) {\n console.log('got %d bytes of data', chunk.length);\n})This event fires when there will be no more data to read.\n\n
\nNote that the end event will not fire unless the data is\ncompletely consumed. This can be done by switching into flowing mode,\nor by calling read() repeatedly until you get to the end.\n\n
var readable = getReadableStreamSomehow();\nreadable.on('data', function(chunk) {\n console.log('got %d bytes of data', chunk.length);\n})\nreadable.on('end', function() {\n console.log('there will be no more data.');\n});Emitted when the underlying resource (for example, the backing file\ndescriptor) has been closed. Not all streams will emit this.\n\n
\n", "params": [] }, { "textRaw": "Event: 'error'", "type": "event", "name": "error", "params": [], "desc": "Emitted if there was an error receiving data.\n\n
\n" } ], "methods": [ { "textRaw": "readable.read([size])", "type": "method", "name": "read", "signatures": [ { "return": { "textRaw": "Return {String | Buffer | null} ", "name": "return", "type": "String | Buffer | null" }, "params": [ { "textRaw": "`size` {Number} Optional argument to specify how much data to read. ", "name": "size", "type": "Number", "desc": "Optional argument to specify how much data to read.", "optional": true } ] }, { "params": [ { "name": "size", "optional": true } ] } ], "desc": "The read() method pulls some data out of the internal buffer and\nreturns it. If there is no data available, then it will return\nnull.\n\n
If you pass in a size argument, then it will return that many\nbytes. If size bytes are not available, then it will return null.\n\n
If you do not specify a size argument, then it will return all the\ndata in the internal buffer.\n\n
This method should only be called in non-flowing mode. In\nflowing-mode, this method is called automatically until the internal\nbuffer is drained.\n\n
\nvar readable = getReadableStreamSomehow();\nreadable.on('readable', function() {\n var chunk;\n while (null !== (chunk = readable.read())) {\n console.log('got %d bytes of data', chunk.length);\n }\n});Call this function to cause the stream to return strings of the\nspecified encoding instead of Buffer objects. For example, if you do\nreadable.setEncoding('utf8'), then the output data will be\ninterpreted as UTF-8 data, and returned as strings. If you do\nreadable.setEncoding('hex'), then the data will be encoded in\nhexadecimal string format.\n\n
This properly handles multi-byte characters that would otherwise be\npotentially mangled if you simply pulled the Buffers directly and\ncalled buf.toString(encoding) on them. If you want to read the data\nas strings, always use this method.\n\n
var readable = getReadableStreamSomehow();\nreadable.setEncoding('utf8');\nreadable.on('data', function(chunk) {\n assert.equal(typeof chunk, 'string');\n console.log('got %d characters of string data', chunk.length);\n})This method will cause the readable stream to resume emitting data\nevents.\n\n
This method will switch the stream into flowing-mode. If you do not\nwant to consume the data from a stream, but you do want to get to\nits end event, you can call readable.resume() to open the flow of\ndata.\n\n
var readable = getReadableStreamSomehow();\nreadable.resume();\nreadable.on('end', function(chunk) {\n console.log('got to the end, but did not read anything');\n})This method will cause a stream in flowing-mode to stop emitting\ndata events. Any data that becomes available will remain in the\ninternal buffer.\n\n
This method is only relevant in flowing mode. When called on a\nnon-flowing stream, it will switch into flowing mode, but remain\npaused.\n\n
\nvar readable = getReadableStreamSomehow();\nreadable.on('data', function(chunk) {\n console.log('got %d bytes of data', chunk.length);\n readable.pause();\n console.log('there will be no more data for 1 second');\n setTimeout(function() {\n console.log('now data will start flowing again');\n readable.resume();\n }, 1000);\n})This method pulls all the data out of a readable stream, and writes it\nto the supplied destination, automatically managing the flow so that\nthe destination is not overwhelmed by a fast readable stream.\n\n
\nMultiple destinations can be piped to safely.\n\n
\nvar readable = getReadableStreamSomehow();\nvar writable = fs.createWriteStream('file.txt');\n// All the data from readable goes into 'file.txt'\nreadable.pipe(writable);This function returns the destination stream, so you can set up pipe\nchains like so:\n\n
\nvar r = fs.createReadStream('file.txt');\nvar z = zlib.createGzip();\nvar w = fs.createWriteStream('file.txt.gz');\nr.pipe(z).pipe(w);For example, emulating the Unix cat command:\n\n
process.stdin.pipe(process.stdout);By default [end()][] is called on the destination when the source stream\nemits end, so that destination is no longer writable. Pass { end:\nfalse } as options to keep the destination stream open.\n\n
This keeps writer open so that "Goodbye" can be written at the\nend.\n\n
reader.pipe(writer, { end: false });\nreader.on('end', function() {\n writer.end('Goodbye\\n');\n});Note that process.stderr and process.stdout are never closed until\nthe process exits, regardless of the specified options.\n\n
This method will remove the hooks set up for a previous pipe() call.\n\n
If the destination is not specified, then all pipes are removed.\n\n
\nIf the destination is specified, but no pipe is set up for it, then\nthis is a no-op.\n\n
\nvar readable = getReadableStreamSomehow();\nvar writable = fs.createWriteStream('file.txt');\n// All the data from readable goes into 'file.txt',\n// but only for the first second\nreadable.pipe(writable);\nsetTimeout(function() {\n console.log('stop writing to file.txt');\n readable.unpipe(writable);\n console.log('manually close the file stream');\n writable.end();\n}, 1000);This is useful in certain cases where a stream is being consumed by a\nparser, which needs to "un-consume" some data that it has\noptimistically pulled out of the source, so that the stream can be\npassed on to some other party.\n\n
\nIf you find that you must often call stream.unshift(chunk) in your\nprograms, consider implementing a [Transform][] stream instead. (See API\nfor Stream Implementors, below.)\n\n
// Pull off a header delimited by \\n\\n\n// use unshift() if we get too much\n// Call the callback with (error, header, stream)\nvar StringDecoder = require('string_decoder').StringDecoder;\nfunction parseHeader(stream, callback) {\n stream.on('error', callback);\n stream.on('readable', onReadable);\n var decoder = new StringDecoder('utf8');\n var header = '';\n function onReadable() {\n var chunk;\n while (null !== (chunk = stream.read())) {\n var str = decoder.write(chunk);\n if (str.match(/\\n\\n/)) {\n // found the header boundary\n var split = str.split(/\\n\\n/);\n header += split.shift();\n var remaining = split.join('\\n\\n');\n var buf = new Buffer(remaining, 'utf8');\n if (buf.length)\n stream.unshift(buf);\n stream.removeListener('error', callback);\n stream.removeListener('readable', onReadable);\n // now the body of the message can be read from the stream.\n callback(null, header, stream);\n } else {\n // still reading the header.\n header += str;\n }\n }\n }\n}Versions of Node prior to v0.10 had streams that did not implement the\nentire Streams API as it is today. (See "Compatibility" below for\nmore information.)\n\n
\nIf you are using an older Node library that emits 'data' events and\nhas a pause() method that is advisory only, then you can use the\nwrap() method to create a [Readable][] stream that uses the old stream\nas its data source.\n\n
You will very rarely ever need to call this function, but it exists\nas a convenience for interacting with old Node programs and libraries.\n\n
\nFor example:\n\n
\nvar OldReader = require('./old-api-module.js').OldReader;\nvar oreader = new OldReader;\nvar Readable = require('stream').Readable;\nvar myReader = new Readable().wrap(oreader);\n\nmyReader.on('readable', function() {\n myReader.read(); // etc.\n});The Writable stream interface is an abstraction for a destination\nthat you are writing data to.\n\n
\nExamples of writable streams include:\n\n
\nThis method writes some data to the underlying system, and calls the\nsupplied callback once the data has been fully handled.\n\n
\nThe return value indicates if you should continue writing right now.\nIf the data had to be buffered internally, then it will return\nfalse. Otherwise, it will return true.\n\n
This return value is strictly advisory. You MAY continue to write,\neven if it returns false. However, writes will be buffered in\nmemory, so it is best not to do this excessively. Instead, wait for\nthe drain event before writing more data.\n\n
Call this method when no more data will be written to the stream. If\nsupplied, the callback is attached as a listener on the finish event.\n\n
Calling [write()][] after calling [end()][] will raise an error.\n\n
// write 'hello, ' and then end with 'world!'\nvar file = fs.createWriteStream('example.txt');\nfile.write('hello, ');\nfile.end('world!');\n// writing more now is not allowed!If a [writable.write(chunk)][] call returns false, then the drain\nevent will indicate when it is appropriate to begin writing more data\nto the stream.\n\n
// Write the data to the supplied writable stream 1MM times.\n// Be attentive to back-pressure.\nfunction writeOneMillionTimes(writer, data, encoding, callback) {\n var i = 1000000;\n write();\n function write() {\n var ok = true;\n do {\n i -= 1;\n if (i === 0) {\n // last time!\n writer.write(data, encoding, callback);\n } else {\n // see if we should continue, or wait\n // don't pass the callback, because we're not done yet.\n ok = writer.write(data, encoding);\n }\n } while (i > 0 && ok);\n if (i > 0) {\n // had to stop early!\n // write some more once it drains\n writer.once('drain', write);\n }\n }\n}When the [end()][] method has been called, and all data has been flushed\nto the underlying system, this event is emitted.\n\n
var writer = getWritableStreamSomehow();\nfor (var i = 0; i < 100; i ++) {\n writer.write('hello, #' + i + '!\\n');\n}\nwriter.end('this is the end\\n');\nwriter.on('finish', function() {\n console.error('all writes are now complete.');\n});This is emitted whenever the pipe() method is called on a readable\nstream, adding this writable to its set of destinations.\n\n
var writer = getWritableStreamSomehow();\nvar reader = getReadableStreamSomehow();\nwriter.on('pipe', function(src) {\n console.error('something is piping into the writer');\n assert.equal(src, reader);\n});\nreader.pipe(writer);This is emitted whenever the [unpipe()][] method is called on a\nreadable stream, removing this writable from its set of destinations.\n\n
var writer = getWritableStreamSomehow();\nvar reader = getReadableStreamSomehow();\nwriter.on('unpipe', function(src) {\n console.error('something has stopped piping into the writer');\n assert.equal(src, reader);\n});\nreader.pipe(writer);\nreader.unpipe(writer);Emitted if there was an error when writing or piping data.\n\n
\n" } ] }, { "textRaw": "Class: stream.Duplex", "type": "class", "name": "stream.Duplex", "desc": "Duplex streams are streams that implement both the [Readable][] and\n[Writable][] interfaces. See above for usage.\n\n
\nExamples of Duplex streams include:\n\n
\nTransform streams are [Duplex][] streams where the output is in some way\ncomputed from the input. They implement both the [Readable][] and\n[Writable][] interfaces. See above for usage.\n\n
\nExamples of Transform streams include:\n\n
\nTo implement any sort of stream, the pattern is the same:\n\n
\nutil.inherits][] method is particularly helpful for this.)The class to extend and the method(s) to implement depend on the sort\nof stream class you are writing:\n\n
\n| \n Use-case \n | \n \n Class \n | \n \n Method(s) to implement \n | \n
|---|---|---|
| \n Reading only \n | \n \n \n | \n\n
| \n
| \n Writing only \n | \n \n \n | \n\n
| \n
| \n Reading and writing \n | \n \n \n | \n\n
| \n
| \n Operate on written data, then read the result \n | \n \n \n | \n\n
| \n
In your implementation code, it is very important to never call the\nmethods described in [API for Stream Consumers][] above. Otherwise, you\ncan potentially cause adverse side effects in programs that consume\nyour streaming interfaces.\n\n
\n", "examples": [ { "textRaw": "Class: stream.Readable", "type": "example", "name": "stream.Readable", "desc": "stream.Readable is an abstract class designed to be extended with an\nunderlying implementation of the [_read(size)][] method.\n\n
Please see above under [API for Stream Consumers][] for how to consume\nstreams in your programs. What follows is an explanation of how to\nimplement Readable streams in your programs.\n\n
\nThis is a basic example of a Readable stream. It emits the numerals\nfrom 1 to 1,000,000 in ascending order, and then ends.\n\n
\nvar Readable = require('stream').Readable;\nvar util = require('util');\nutil.inherits(Counter, Readable);\n\nfunction Counter(opt) {\n Readable.call(this, opt);\n this._max = 1000000;\n this._index = 1;\n}\n\nCounter.prototype._read = function() {\n var i = this._index++;\n if (i > this._max)\n this.push(null);\n else {\n var str = '' + i;\n var buf = new Buffer(str, 'ascii');\n this.push(buf);\n }\n};This is similar to the parseHeader function described above, but\nimplemented as a custom stream. Also, note that this implementation\ndoes not convert the incoming data to a string.\n\n
However, this would be better implemented as a [Transform][] stream. See\nbelow for a better implementation.\n\n
\n// A parser for a simple data protocol.\n// The "header" is a JSON object, followed by 2 \\n characters, and\n// then a message body.\n//\n// NOTE: This can be done more simply as a Transform stream!\n// Using Readable directly for this is sub-optimal. See the\n// alternative example below under the Transform section.\n\nvar Readable = require('stream').Readable;\nvar util = require('util');\n\nutil.inherits(SimpleProtocol, Readable);\n\nfunction SimpleProtocol(source, options) {\n if (!(this instanceof SimpleProtocol))\n return new SimpleProtocol(source, options);\n\n Readable.call(this, options);\n this._inBody = false;\n this._sawFirstCr = false;\n\n // source is a readable stream, such as a socket or file\n this._source = source;\n\n var self = this;\n source.on('end', function() {\n self.push(null);\n });\n\n // give it a kick whenever the source is readable\n // read(0) will not consume any bytes\n source.on('readable', function() {\n self.read(0);\n });\n\n this._rawHeader = [];\n this.header = null;\n}\n\nSimpleProtocol.prototype._read = function(n) {\n if (!this._inBody) {\n var chunk = this._source.read();\n\n // if the source doesn't have data, we don't have data yet.\n if (chunk === null)\n return this.push('');\n\n // check if the chunk has a \\n\\n\n var split = -1;\n for (var i = 0; i < chunk.length; i++) {\n if (chunk[i] === 10) { // '\\n'\n if (this._sawFirstCr) {\n split = i;\n break;\n } else {\n this._sawFirstCr = true;\n }\n } else {\n this._sawFirstCr = false;\n }\n }\n\n if (split === -1) {\n // still waiting for the \\n\\n\n // stash the chunk, and try again.\n this._rawHeader.push(chunk);\n this.push('');\n } else {\n this._inBody = true;\n var h = chunk.slice(0, split);\n this._rawHeader.push(h);\n var header = Buffer.concat(this._rawHeader).toString();\n try {\n this.header = JSON.parse(header);\n } catch (er) {\n this.emit('error', new Error('invalid simple protocol data'));\n return;\n }\n // now, because we got some extra data, unshift the rest\n // back into the read queue so that our consumer will see it.\n var b = chunk.slice(split);\n this.unshift(b);\n\n // and let them know that we are done parsing the header.\n this.emit('header', this.header);\n }\n } else {\n // from there on, just provide the data to our consumer.\n // careful not to push(null), since that would indicate EOF.\n var chunk = this._source.read();\n if (chunk) this.push(chunk);\n }\n};\n\n// Usage:\n// var parser = new SimpleProtocol(source);\n// Now parser is a readable stream that will emit 'header'\n// with the parsed header data.In classes that extend the Readable class, make sure to call the\nReadable constructor so that the buffering settings can be properly\ninitialized.\n\n
\n" }, { "textRaw": "readable.\\_read(size)", "type": "method", "name": "\\_read", "signatures": [ { "params": [ { "textRaw": "`size` {Number} Number of bytes to read asynchronously ", "name": "size", "type": "Number", "desc": "Number of bytes to read asynchronously" } ] }, { "params": [ { "name": "size" } ] } ], "desc": "Note: Implement this function, but do NOT call it directly.\n\n
\nThis function should NOT be called directly. It should be implemented\nby child classes, and only called by the internal Readable class\nmethods.\n\n
\nAll Readable stream implementations must provide a _read method to\nfetch data from the underlying resource.\n\n
This method is prefixed with an underscore because it is internal to\nthe class that defines it, and should not be called directly by user\nprograms. However, you are expected to override this method in\nyour own extension classes.\n\n
\nWhen data is available, put it into the read queue by calling\nreadable.push(chunk). If push returns false, then you should stop\nreading. When _read is called again, you should start pushing more\ndata.\n\n
The size argument is advisory. Implementations where a "read" is a\nsingle call that returns data can use this to know how much data to\nfetch. Implementations where that is not relevant, such as TCP or\nTLS, may ignore this argument, and simply provide data whenever it\nbecomes available. There is no need, for example to "wait" until\nsize bytes are available before calling [stream.push(chunk)][].\n\n
Note: This function should be called by Readable implementors, NOT\nby consumers of Readable streams.\n\n
\nThe _read() function will not be called again until at least one\npush(chunk) call is made.\n\n
The Readable class works by putting data into a read queue to be\npulled out later by calling the read() method when the 'readable'\nevent fires.\n\n
The push() method will explicitly insert some data into the read\nqueue. If it is called with null then it will signal the end of the\ndata (EOF).\n\n
This API is designed to be as flexible as possible. For example,\nyou may be wrapping a lower-level source which has some sort of\npause/resume mechanism, and a data callback. In those cases, you\ncould wrap the low-level source object by doing something like this:\n\n
\n// source is an object with readStop() and readStart() methods,\n// and an `ondata` member that gets called when it has data, and\n// an `onend` member that gets called when the data is over.\n\nutil.inherits(SourceWrapper, Readable);\n\nfunction SourceWrapper(options) {\n Readable.call(this, options);\n\n this._source = getLowlevelSourceObject();\n var self = this;\n\n // Every time there's data, we push it into the internal buffer.\n this._source.ondata = function(chunk) {\n // if push() returns false, then we need to stop reading from source\n if (!self.push(chunk))\n self._source.readStop();\n };\n\n // When the source ends, we push the EOF-signalling `null` chunk\n this._source.onend = function() {\n self.push(null);\n };\n}\n\n// _read will be called when the stream wants to pull more data in\n// the advisory size argument is ignored in this case.\nSourceWrapper.prototype._read = function(size) {\n this._source.readStart();\n};stream.Writable is an abstract class designed to be extended with an\nunderlying implementation of the [_write(chunk, encoding, callback)][] method.\n\n
Please see above under [API for Stream Consumers][] for how to consume\nwritable streams in your programs. What follows is an explanation of\nhow to implement Writable streams in your programs.\n\n
\n", "methods": [ { "textRaw": "new stream.Writable([options])", "type": "method", "name": "Writable", "signatures": [ { "params": [ { "textRaw": "`options` {Object} ", "options": [ { "textRaw": "`highWaterMark` {Number} Buffer level when [`write()`][] starts returning false. Default=16kb ", "name": "highWaterMark", "type": "Number", "desc": "Buffer level when [`write()`][] starts returning false. Default=16kb" }, { "textRaw": "`decodeStrings` {Boolean} Whether or not to decode strings into Buffers before passing them to [`_write()`][]. Default=true ", "name": "decodeStrings", "type": "Boolean", "desc": "Whether or not to decode strings into Buffers before passing them to [`_write()`][]. Default=true" }, { "textRaw": "`objectMode` {Boolean} Whether or not the `write(anyObj)` is a valid operation. If set you can write arbitrary data instead of only `Buffer` / `String` data. Default=false ", "name": "objectMode", "type": "Boolean", "desc": "Whether or not the `write(anyObj)` is a valid operation. If set you can write arbitrary data instead of only `Buffer` / `String` data. Default=false" } ], "name": "options", "type": "Object", "optional": true } ] }, { "params": [ { "name": "options", "optional": true } ] } ], "desc": "In classes that extend the Writable class, make sure to call the\nconstructor so that the buffering settings can be properly\ninitialized.\n\n
\n" }, { "textRaw": "writable.\\_write(chunk, encoding, callback)", "type": "method", "name": "\\_write", "signatures": [ { "params": [ { "textRaw": "`chunk` {Buffer | String} The chunk to be written. Will always be a buffer unless the `decodeStrings` option was set to `false`. ", "name": "chunk", "type": "Buffer | String", "desc": "The chunk to be written. Will always be a buffer unless the `decodeStrings` option was set to `false`." }, { "textRaw": "`encoding` {String} If the chunk is a string, then this is the encoding type. Ignore chunk is a buffer. Note that chunk will **always** be a buffer unless the `decodeStrings` option is explicitly set to `false`. ", "name": "encoding", "type": "String", "desc": "If the chunk is a string, then this is the encoding type. Ignore chunk is a buffer. Note that chunk will **always** be a buffer unless the `decodeStrings` option is explicitly set to `false`." }, { "textRaw": "`callback` {Function} Call this function (optionally with an error argument) when you are done processing the supplied chunk. ", "name": "callback", "type": "Function", "desc": "Call this function (optionally with an error argument) when you are done processing the supplied chunk." } ] }, { "params": [ { "name": "chunk" }, { "name": "encoding" }, { "name": "callback" } ] } ], "desc": "All Writable stream implementations must provide a [_write()][]\nmethod to send data to the underlying resource.\n\n
Note: This function MUST NOT be called directly. It should be\nimplemented by child classes, and called by the internal Writable\nclass methods only.\n\n
\nCall the callback using the standard callback(error) pattern to\nsignal that the write completed successfully or with an error.\n\n
If the decodeStrings flag is set in the constructor options, then\nchunk may be a string rather than a Buffer, and encoding will\nindicate the sort of string that it is. This is to support\nimplementations that have an optimized handling for certain string\ndata encodings. If you do not explicitly set the decodeStrings\noption to false, then you can safely ignore the encoding argument,\nand assume that chunk will always be a Buffer.\n\n
This method is prefixed with an underscore because it is internal to\nthe class that defines it, and should not be called directly by user\nprograms. However, you are expected to override this method in\nyour own extension classes.\n\n\n
\n" } ] }, { "textRaw": "Class: stream.Duplex", "type": "class", "name": "stream.Duplex", "desc": "A "duplex" stream is one that is both Readable and Writable, such as a\nTCP socket connection.\n\n
\nNote that stream.Duplex is an abstract class designed to be extended\nwith an underlying implementation of the _read(size) and\n[_write(chunk, encoding, callback)][] methods as you would with a\nReadable or Writable stream class.\n\n
Since JavaScript doesn't have multiple prototypal inheritance, this\nclass prototypally inherits from Readable, and then parasitically from\nWritable. It is thus up to the user to implement both the lowlevel\n_read(n) method as well as the lowlevel\n[_write(chunk, encoding, callback)][] method on extension duplex classes.\n\n
In classes that extend the Duplex class, make sure to call the\nconstructor so that the buffering settings can be properly\ninitialized.\n\n\n
\n" } ] }, { "textRaw": "Class: stream.Transform", "type": "class", "name": "stream.Transform", "desc": "A "transform" stream is a duplex stream where the output is causally\nconnected in some way to the input, such as a [zlib][] stream or a\n[crypto][] stream.\n\n
\nThere is no requirement that the output be the same size as the input,\nthe same number of chunks, or arrive at the same time. For example, a\nHash stream will only ever have a single chunk of output which is\nprovided when the input is ended. A zlib stream will produce output\nthat is either much smaller or much larger than its input.\n\n
\nRather than implement the [_read()][] and [_write()][] methods, Transform\nclasses must implement the _transform() method, and may optionally\nalso implement the _flush() method. (See below.)\n\n
In classes that extend the Transform class, make sure to call the\nconstructor so that the buffering settings can be properly\ninitialized.\n\n
\n" }, { "textRaw": "transform.\\_transform(chunk, encoding, callback)", "type": "method", "name": "\\_transform", "signatures": [ { "params": [ { "textRaw": "`chunk` {Buffer | String} The chunk to be transformed. Will always be a buffer unless the `decodeStrings` option was set to `false`. ", "name": "chunk", "type": "Buffer | String", "desc": "The chunk to be transformed. Will always be a buffer unless the `decodeStrings` option was set to `false`." }, { "textRaw": "`encoding` {String} If the chunk is a string, then this is the encoding type. (Ignore if `decodeStrings` chunk is a buffer.) ", "name": "encoding", "type": "String", "desc": "If the chunk is a string, then this is the encoding type. (Ignore if `decodeStrings` chunk is a buffer.)" }, { "textRaw": "`callback` {Function} Call this function (optionally with an error argument and data) when you are done processing the supplied chunk. ", "name": "callback", "type": "Function", "desc": "Call this function (optionally with an error argument and data) when you are done processing the supplied chunk." } ] }, { "params": [ { "name": "chunk" }, { "name": "encoding" }, { "name": "callback" } ] } ], "desc": "Note: This function MUST NOT be called directly. It should be\nimplemented by child classes, and called by the internal Transform\nclass methods only.\n\n
\nAll Transform stream implementations must provide a _transform\nmethod to accept input and produce output.\n\n
_transform should do whatever has to be done in this specific\nTransform class, to handle the bytes being written, and pass them off\nto the readable portion of the interface. Do asynchronous I/O,\nprocess things, and so on.\n\n
Call transform.push(outputChunk) 0 or more times to generate output\nfrom this input chunk, depending on how much data you want to output\nas a result of this chunk.\n\n
Call the callback function only when the current chunk is completely\nconsumed. Note that there may or may not be output as a result of any\nparticular input chunk. If you supply as the second argument to the\nit will be passed to push method, in other words the following are\nequivalent:\n\n
\ntransform.prototype._transform = function (data, encoding, callback) {\n this.push(data);\n callback();\n}\n\ntransform.prototype._transform = function (data, encoding, callback) {\n callback(null, data);\n}This method is prefixed with an underscore because it is internal to\nthe class that defines it, and should not be called directly by user\nprograms. However, you are expected to override this method in\nyour own extension classes.\n\n
\n" }, { "textRaw": "transform.\\_flush(callback)", "type": "method", "name": "\\_flush", "signatures": [ { "params": [ { "textRaw": "`callback` {Function} Call this function (optionally with an error argument) when you are done flushing any remaining data. ", "name": "callback", "type": "Function", "desc": "Call this function (optionally with an error argument) when you are done flushing any remaining data." } ] }, { "params": [ { "name": "callback" } ] } ], "desc": "Note: This function MUST NOT be called directly. It MAY be implemented\nby child classes, and if so, will be called by the internal Transform\nclass methods only.\n\n
\nIn some cases, your transform operation may need to emit a bit more\ndata at the end of the stream. For example, a Zlib compression\nstream will store up some internal state so that it can optimally\ncompress the output. At the end, however, it needs to do the best it\ncan with what is left, so that the data will be complete.\n\n
In those cases, you can implement a _flush method, which will be\ncalled at the very end, after all the written data is consumed, but\nbefore emitting end to signal the end of the readable side. Just\nlike with _transform, call transform.push(chunk) zero or more\ntimes, as appropriate, and call callback when the flush operation is\ncomplete.\n\n
This method is prefixed with an underscore because it is internal to\nthe class that defines it, and should not be called directly by user\nprograms. However, you are expected to override this method in\nyour own extension classes.\n\n
\n" } ], "modules": [ { "textRaw": "Events: 'finish' and 'end'", "name": "events:_'finish'_and_'end'", "desc": "The [finish][] and [end][] events are from the parent Writable\nand Readable classes respectively. The finish event is fired after\n.end() is called and all chunks have been processed by _transform,\nend is fired after all data has been output which is after the callback\nin _flush has been called.\n\n
SimpleProtocol parser v2The example above of a simple protocol parser can be implemented\nsimply by using the higher level [Transform][] stream class, similar to\nthe parseHeader and SimpleProtocol v1 examples above.\n\n
In this example, rather than providing the input as an argument, it\nwould be piped into the parser, which is a more idiomatic Node stream\napproach.\n\n
\nvar util = require('util');\nvar Transform = require('stream').Transform;\nutil.inherits(SimpleProtocol, Transform);\n\nfunction SimpleProtocol(options) {\n if (!(this instanceof SimpleProtocol))\n return new SimpleProtocol(options);\n\n Transform.call(this, options);\n this._inBody = false;\n this._sawFirstCr = false;\n this._rawHeader = [];\n this.header = null;\n}\n\nSimpleProtocol.prototype._transform = function(chunk, encoding, done) {\n if (!this._inBody) {\n // check if the chunk has a \\n\\n\n var split = -1;\n for (var i = 0; i < chunk.length; i++) {\n if (chunk[i] === 10) { // '\\n'\n if (this._sawFirstCr) {\n split = i;\n break;\n } else {\n this._sawFirstCr = true;\n }\n } else {\n this._sawFirstCr = false;\n }\n }\n\n if (split === -1) {\n // still waiting for the \\n\\n\n // stash the chunk, and try again.\n this._rawHeader.push(chunk);\n } else {\n this._inBody = true;\n var h = chunk.slice(0, split);\n this._rawHeader.push(h);\n var header = Buffer.concat(this._rawHeader).toString();\n try {\n this.header = JSON.parse(header);\n } catch (er) {\n this.emit('error', new Error('invalid simple protocol data'));\n return;\n }\n // and let them know that we are done parsing the header.\n this.emit('header', this.header);\n\n // now, because we got some extra data, emit this first.\n this.push(chunk.slice(split));\n }\n } else {\n // from there on, just provide the data to our consumer as-is.\n this.push(chunk);\n }\n done();\n};\n\n// Usage:\n// var parser = new SimpleProtocol();\n// source.pipe(parser)\n// Now parser is a readable stream that will emit 'header'\n// with the parsed header data.This is a trivial implementation of a [Transform][] stream that simply\npasses the input bytes across to the output. Its purpose is mainly\nfor examples and testing, but there are occasionally use cases where\nit can come in handy as a building block for novel sorts of streams.\n\n\n
\n" } ] }, { "textRaw": "Streams: Under the Hood", "name": "Streams: Under the Hood", "type": "misc", "miscs": [ { "textRaw": "Buffering", "name": "Buffering", "type": "misc", "desc": "Both Writable and Readable streams will buffer data on an internal\nobject called _writableState.buffer or _readableState.buffer,\nrespectively.\n\n
The amount of data that will potentially be buffered depends on the\nhighWaterMark option which is passed into the constructor.\n\n
Buffering in Readable streams happens when the implementation calls\n[stream.push(chunk)][]. If the consumer of the Stream does not call\nstream.read(), then the data will sit in the internal queue until it\nis consumed.\n\n
Buffering in Writable streams happens when the user calls\n[stream.write(chunk)][] repeatedly, even when write() returns false.\n\n
The purpose of streams, especially with the pipe() method, is to\nlimit the buffering of data to acceptable levels, so that sources and\ndestinations of varying speed will not overwhelm the available memory.\n\n
There are some cases where you want to trigger a refresh of the\nunderlying readable stream mechanisms, without actually consuming any\ndata. In that case, you can call stream.read(0), which will always\nreturn null.\n\n
If the internal read buffer is below the highWaterMark, and the\nstream is not currently reading, then calling read(0) will trigger\na low-level _read call.\n\n
There is almost never a need to do this. However, you will see some\ncases in Node's internals where this is done, particularly in the\nReadable stream class internals.\n\n
\n", "type": "misc", "displayName": "`stream.read(0)`" }, { "textRaw": "`stream.push('')`", "name": "`stream.push('')`", "desc": "Pushing a zero-byte string or Buffer (when not in [Object mode][]) has an\ninteresting side effect. Because it is a call to\n[stream.push()][], it will end the reading process. However, it\ndoes not add any data to the readable buffer, so there's nothing for\na user to consume.\n\n
Very rarely, there are cases where you have no data to provide now,\nbut the consumer of your stream (or, perhaps, another bit of your own\ncode) will know when to check again, by calling stream.read(0). In\nthose cases, you may call stream.push('').\n\n
So far, the only use case for this functionality is in the\n[tls.CryptoStream][] class, which is deprecated in Node v0.12. If you\nfind that you have to use stream.push(''), please consider another\napproach, because it almost certainly indicates that something is\nhorribly wrong.\n\n
In versions of Node prior to v0.10, the Readable stream interface was\nsimpler, but also less powerful and less useful.\n\n
\nread() method, 'data'\nevents would start emitting immediately. If you needed to do some\nI/O to decide how to handle data, then you had to store the chunks\nin some kind of buffer so that they would not be lost.pause() method was advisory, rather than guaranteed. This\nmeant that you still had to be prepared to receive 'data' events\neven when the stream was in a paused state.In Node v0.10, the Readable class described below was added. For\nbackwards compatibility with older Node programs, Readable streams\nswitch into "flowing mode" when a 'data' event handler is added, or\nwhen the pause() or resume() methods are called. The effect is\nthat, even if you are not using the new read() method and\n'readable' event, you no longer have to worry about losing 'data'\nchunks.\n\n
Most programs will continue to function normally. However, this\nintroduces an edge case in the following conditions:\n\n
\n'data' event handler is added.pause() and resume() methods are never called.For example, consider the following code:\n\n
\n// WARNING! BROKEN!\nnet.createServer(function(socket) {\n\n // we add an 'end' method, but never consume the data\n socket.on('end', function() {\n // It will never get here.\n socket.end('I got your message (but didnt read it)\\n');\n });\n\n}).listen(1337);In versions of node prior to v0.10, the incoming message data would be\nsimply discarded. However, in Node v0.10 and beyond, the socket will\nremain paused forever.\n\n
\nThe workaround in this situation is to call the resume() method to\ntrigger "old mode" behavior:\n\n
// Workaround\nnet.createServer(function(socket) {\n\n socket.on('end', function() {\n socket.end('I got your message (but didnt read it)\\n');\n });\n\n // start the flow of data, discarding it.\n socket.resume();\n\n}).listen(1337);In addition to new Readable streams switching into flowing-mode, pre-v0.10\nstyle streams can be wrapped in a Readable class using the wrap()\nmethod.\n\n\n
Normally, Streams operate on Strings and Buffers exclusively.\n\n
\nStreams that are in object mode can emit generic JavaScript values\nother than Buffers and Strings.\n\n
\nA Readable stream in object mode will always return a single item from\na call to stream.read(size), regardless of what the size argument\nis.\n\n
A Writable stream in object mode will always ignore the encoding\nargument to stream.write(data, encoding).\n\n
The special value null still retains its special value for object\nmode streams. That is, for object mode readable streams, null as a\nreturn value from stream.read() indicates that there is no more\ndata, and [stream.push(null)][] will signal the end of stream data\n(EOF).\n\n
No streams in Node core are object mode streams. This pattern is only\nused by userland streaming libraries.\n\n
\nYou should set objectMode in your stream child class constructor on\nthe options object. Setting objectMode mid-stream is not safe.\n\n
[Readable][] streams have a member object called _readableState.\n[Writable][] streams have a member object called _writableState.\n[Duplex][] streams have both.\n\n
These objects should generally not be modified in child classes.\nHowever, if you have a Duplex or Transform stream that should be in\nobjectMode on the readable side, and not in objectMode on the\nwritable side, then you may do this in the constructor by setting the\nflag explicitly on the appropriate state object.\n\n
var util = require('util');\nvar StringDecoder = require('string_decoder').StringDecoder;\nvar Transform = require('stream').Transform;\nutil.inherits(JSONParseStream, Transform);\n\n// Gets \\n-delimited JSON string data, and emits the parsed objects\nfunction JSONParseStream(options) {\n if (!(this instanceof JSONParseStream))\n return new JSONParseStream(options);\n\n Transform.call(this, options);\n this._writableState.objectMode = false;\n this._readableState.objectMode = true;\n this._buffer = '';\n this._decoder = new StringDecoder('utf8');\n}\n\nJSONParseStream.prototype._transform = function(chunk, encoding, cb) {\n this._buffer += this._decoder.write(chunk);\n // split on newlines\n var lines = this._buffer.split(/\\r?\\n/);\n // keep the last partial line buffered\n this._buffer = lines.pop();\n for (var l = 0; l < lines.length; l++) {\n var line = lines[l];\n try {\n var obj = JSON.parse(line);\n } catch (er) {\n this.emit('error', er);\n return;\n }\n // push the parsed object out to the readable consumer\n this.push(obj);\n }\n cb();\n};\n\nJSONParseStream.prototype._flush = function(cb) {\n // Just handle any leftover\n var rem = this._buffer.trim();\n if (rem) {\n try {\n var obj = JSON.parse(rem);\n } catch (er) {\n this.emit('error', er);\n return;\n }\n // push the parsed object out to the readable consumer\n this.push(obj);\n }\n cb();\n};The state objects contain other useful information for debugging the\nstate of streams in your programs. It is safe to look at them, but\nbeyond setting option flags in the constructor, it is not safe to\nmodify them.\n\n\n
\n", "type": "misc", "displayName": "State Objects" } ] } ], "examples": [ { "textRaw": "Class: stream.Readable", "type": "example", "name": "stream.Readable", "desc": "stream.Readable is an abstract class designed to be extended with an\nunderlying implementation of the [_read(size)][] method.\n\n
Please see above under [API for Stream Consumers][] for how to consume\nstreams in your programs. What follows is an explanation of how to\nimplement Readable streams in your programs.\n\n
\nThis is a basic example of a Readable stream. It emits the numerals\nfrom 1 to 1,000,000 in ascending order, and then ends.\n\n
\nvar Readable = require('stream').Readable;\nvar util = require('util');\nutil.inherits(Counter, Readable);\n\nfunction Counter(opt) {\n Readable.call(this, opt);\n this._max = 1000000;\n this._index = 1;\n}\n\nCounter.prototype._read = function() {\n var i = this._index++;\n if (i > this._max)\n this.push(null);\n else {\n var str = '' + i;\n var buf = new Buffer(str, 'ascii');\n this.push(buf);\n }\n};This is similar to the parseHeader function described above, but\nimplemented as a custom stream. Also, note that this implementation\ndoes not convert the incoming data to a string.\n\n
However, this would be better implemented as a [Transform][] stream. See\nbelow for a better implementation.\n\n
\n// A parser for a simple data protocol.\n// The "header" is a JSON object, followed by 2 \\n characters, and\n// then a message body.\n//\n// NOTE: This can be done more simply as a Transform stream!\n// Using Readable directly for this is sub-optimal. See the\n// alternative example below under the Transform section.\n\nvar Readable = require('stream').Readable;\nvar util = require('util');\n\nutil.inherits(SimpleProtocol, Readable);\n\nfunction SimpleProtocol(source, options) {\n if (!(this instanceof SimpleProtocol))\n return new SimpleProtocol(source, options);\n\n Readable.call(this, options);\n this._inBody = false;\n this._sawFirstCr = false;\n\n // source is a readable stream, such as a socket or file\n this._source = source;\n\n var self = this;\n source.on('end', function() {\n self.push(null);\n });\n\n // give it a kick whenever the source is readable\n // read(0) will not consume any bytes\n source.on('readable', function() {\n self.read(0);\n });\n\n this._rawHeader = [];\n this.header = null;\n}\n\nSimpleProtocol.prototype._read = function(n) {\n if (!this._inBody) {\n var chunk = this._source.read();\n\n // if the source doesn't have data, we don't have data yet.\n if (chunk === null)\n return this.push('');\n\n // check if the chunk has a \\n\\n\n var split = -1;\n for (var i = 0; i < chunk.length; i++) {\n if (chunk[i] === 10) { // '\\n'\n if (this._sawFirstCr) {\n split = i;\n break;\n } else {\n this._sawFirstCr = true;\n }\n } else {\n this._sawFirstCr = false;\n }\n }\n\n if (split === -1) {\n // still waiting for the \\n\\n\n // stash the chunk, and try again.\n this._rawHeader.push(chunk);\n this.push('');\n } else {\n this._inBody = true;\n var h = chunk.slice(0, split);\n this._rawHeader.push(h);\n var header = Buffer.concat(this._rawHeader).toString();\n try {\n this.header = JSON.parse(header);\n } catch (er) {\n this.emit('error', new Error('invalid simple protocol data'));\n return;\n }\n // now, because we got some extra data, unshift the rest\n // back into the read queue so that our consumer will see it.\n var b = chunk.slice(split);\n this.unshift(b);\n\n // and let them know that we are done parsing the header.\n this.emit('header', this.header);\n }\n } else {\n // from there on, just provide the data to our consumer.\n // careful not to push(null), since that would indicate EOF.\n var chunk = this._source.read();\n if (chunk) this.push(chunk);\n }\n};\n\n// Usage:\n// var parser = new SimpleProtocol(source);\n// Now parser is a readable stream that will emit 'header'\n// with the parsed header data.In classes that extend the Readable class, make sure to call the\nReadable constructor so that the buffering settings can be properly\ninitialized.\n\n
\n" }, { "textRaw": "readable.\\_read(size)", "type": "method", "name": "\\_read", "signatures": [ { "params": [ { "textRaw": "`size` {Number} Number of bytes to read asynchronously ", "name": "size", "type": "Number", "desc": "Number of bytes to read asynchronously" } ] }, { "params": [ { "name": "size" } ] } ], "desc": "Note: Implement this function, but do NOT call it directly.\n\n
\nThis function should NOT be called directly. It should be implemented\nby child classes, and only called by the internal Readable class\nmethods.\n\n
\nAll Readable stream implementations must provide a _read method to\nfetch data from the underlying resource.\n\n
This method is prefixed with an underscore because it is internal to\nthe class that defines it, and should not be called directly by user\nprograms. However, you are expected to override this method in\nyour own extension classes.\n\n
\nWhen data is available, put it into the read queue by calling\nreadable.push(chunk). If push returns false, then you should stop\nreading. When _read is called again, you should start pushing more\ndata.\n\n
The size argument is advisory. Implementations where a "read" is a\nsingle call that returns data can use this to know how much data to\nfetch. Implementations where that is not relevant, such as TCP or\nTLS, may ignore this argument, and simply provide data whenever it\nbecomes available. There is no need, for example to "wait" until\nsize bytes are available before calling [stream.push(chunk)][].\n\n
Note: This function should be called by Readable implementors, NOT\nby consumers of Readable streams.\n\n
\nThe _read() function will not be called again until at least one\npush(chunk) call is made.\n\n
The Readable class works by putting data into a read queue to be\npulled out later by calling the read() method when the 'readable'\nevent fires.\n\n
The push() method will explicitly insert some data into the read\nqueue. If it is called with null then it will signal the end of the\ndata (EOF).\n\n
This API is designed to be as flexible as possible. For example,\nyou may be wrapping a lower-level source which has some sort of\npause/resume mechanism, and a data callback. In those cases, you\ncould wrap the low-level source object by doing something like this:\n\n
\n// source is an object with readStop() and readStart() methods,\n// and an `ondata` member that gets called when it has data, and\n// an `onend` member that gets called when the data is over.\n\nutil.inherits(SourceWrapper, Readable);\n\nfunction SourceWrapper(options) {\n Readable.call(this, options);\n\n this._source = getLowlevelSourceObject();\n var self = this;\n\n // Every time there's data, we push it into the internal buffer.\n this._source.ondata = function(chunk) {\n // if push() returns false, then we need to stop reading from source\n if (!self.push(chunk))\n self._source.readStop();\n };\n\n // When the source ends, we push the EOF-signalling `null` chunk\n this._source.onend = function() {\n self.push(null);\n };\n}\n\n// _read will be called when the stream wants to pull more data in\n// the advisory size argument is ignored in this case.\nSourceWrapper.prototype._read = function(size) {\n this._source.readStart();\n};