javascript - Node.js streams and data disappearing -

I am playing with the readable and transforming streams, and I can not solve the mystery of missing lines.

Consider a text file that has sequential numbers in lines, from 1 to 20000:

  $ seq 1 20000> File.txt  

I created a readable stream and a linestream (from a library named byline ) I am: npm installed biointe ; I'm using version 4.1.1):

  var file = (required ('FS')). CreateReadStream ('file.txt'); Var line = new (required ('bineline'). Linestream) ();  

Consider the following code:

  setTimeout (function () {lines.on ('readable', function) (var line; while (null ! == (line = lines. Read ()) {console.log (line);}}};}, 1500); SetTimeout (function () (file.on ('readable', function () {var chunk; while (null! == (chunk = file.read ())) {lines.write (piece);}});} , 1000);  

Note that this is the first time the 'readable' event file readable stream, which is the lines stream, and only half a second later it connects a listener to the 'readable' event to the line stream, which prints lines only for the console

If I run this code, then it is only 16384 (which is 2 ^ 14) lines and stops This file will not end, however, if I change the 1500 mms timeout to 500 mms - effectively swap the order in which the audience is attached, it will happily print the entire file.

I have tried to play with the high watermark,

What can be understood about this behavior?

Thank you!

< / P>

/ div>

I think this behavior Can be explained with things:

1. you how to use the streams
2. how byline function.

The problem with the manual piping that you do is that it does not respect the high watermark and forces the buffer to complete.

All these reasons to behave badly due to byline . Look at this: This means that it stops pushing the lines, when buffer lengths> high watermarks but it does not make any sense! This does not prevent memory usage development (lines are still stored in special line buffers), but the stream does not know about these lines and if it ends in overflow mode, they will be lost forever.

Can you do this:

1. Use pipes
2. Modify highWaterMark : lines._readableState.highWaterMark = Infinity; Stop using byline