"ptw" Command

How to Enable Line-buffering Mode Even for commands with Which stdbuf Doesn't Work, Such as Perl, Python

Japanese document is here.（日本語版はこちら）

To Go Straight to the Bottom Line

1. Download this

https://github.com/ShellShoccar-jpn/misc-tools/blob/master/C_SRC/ptw.c or the direct link

2. Compile it as follows:

$ cc -O3 -o ptw ptw.c

Then, the command named "ptw" (pseudo terminal wrapper) will be born.

3. Try and compare the following commands to make sure ptw works correctly.

# Failure (with stdbuf)
$ (i=0; while [ $i -lt 2 ]; do echo $i; sleep 1; i=$((i+1)); done; echo $i) | stdbuf -o L perl -pe 's///' | cat

# Success (with this command)
$ (i=0; while [ $i -lt 2 ]; do echo $i; sleep 1; i=$((i+1)); done; echo $i) | ./ptw perl -pe 's///' | cat

The Reason Why stdbuf Fails to Change the Buffering Mode

Most of commands on which stdbuf doesn't work set buffering mode themselves. And almost all of them give us some ways to change the buffering mode. Here are some examples.

• Perl
  • Set the built-in variable "$|" to 1 if you want Perl to work in Line-buffering mode.
• Python
  • If you execute python with the "-u" option when starting it, it doesn't buffer data at all.

The timing when stdbuf command switches the buffering mode is before starting the target command. Thus, stdbuf switches the buffering mode at first and start the command.

Therefore, the buffering mode that stdbuf set is overwritten if the target command initializes the mode on its own after executed.

The Reason Why ptw Can Change the Buffering Mode Even for Such Commands

The functions of the <stdio.h> library decide the default buffering mode depending on where the command which includes its library is located. In the case which it is in the head or middle of the pipeline, the default is in full-buffering mode. In the case which is in the terminal, line-buffering mode. The commands which manage buffering mode on their own also follow the custom.

So, ptw command cheats the target command with a pseudo-terminal (PTY). At first, ptw makes a PTY, which is connected to the standard output. Then, ptw replaces the standard output for the target with the PTY, and finally executes the target command. The target command assumes that the target itself is located at the end of the pipeline because the standard output seems to be a terminal. As a result, the target command sends data in line buffering mode. That is the mechanism of ptw.

Limitations

Even this command, it is not effective against such as the following commands.

• Commands which don't use the <stdio.h> library. That is because ptw is to cheat the <stdio.h> buffer and the like with their habits in the first place.
• Commands which don't care where they are in the pipeline and always use the full-buffering mode even though they use <stdio.h>. For example, mawk is one of them. It always works on full-buffering mode unless the option "-W interactive" is given.