I have to start by saying that bash bugs me. ;-) I miss having a real /bin/sh – like on NetBSD and FreeBSD.

Ok, got that off my chest.

My dynamic DNS updater code runs a script, once a week, that appends a single character to a file. Another script, the normal “what’s my IP address now" script that runs every 10 minutes, also checks to see if the weeks file contains the four characters “1111” – in which case 28 days have elapsed, and I need to send a special “keepalive” request to dynDNS to keep my account going.

Yes, I’m counting in unary. It’s incrediby useful sometimes!

The scripts both ran fine on my Arch Linux box, but testing them on my Mac I discovered weird junk in my weeks file. The script executes

  echo -n "1" >> weeks

but instead of getting one character longer the file was getting five characters longer. The first time it runs, weeks contains the string

  "-n 1\n"

which, including the trailing newline (that the "-n” was supposed to suppress!), is five characters long. (Those quotes are there so you can see where the string begins and ends, and are not part of the string! Like, you know, normal quotes. ;-)

I tried this over and over and thought I was crazy. Then I went online and found an account of the bug, so I thought I would join the fun and document it here as well.

Here’s the bug: if bash is started as sh, echo -n is broken. It echoes the "-n” and appends a newline.

I’m running the version of bash that’s shipped with OSX 10.5, and I’ve updated (or been updated ;-) to 10.5.8. The version of bash on my Mac says this:

  GNU bash, version 3.2.17(1)-release (i386-apple-darwin9.0)

And it’s broken.

But this Arch Linux version

  GNU bash, version 3.2.48(1)-release (i686-pc-linux-gnu)

works fine.

My “fix” was to call /bin/echo – the actual binary – rather than rely on the built-in echo, which, as we now know, is broken.

I could also have called the script with bash with a bangpath like

  #!/bin/bash

but this is a non-portable Linuxism, and having seen the light (by running BSD) I now realise this. I want to call my script with /bin/sh, and running /bin/echo seems to work ... and this should work on all Linux & BSD systems too.

I wish people in the Linux world knew there was a difference between bash and sh. Having two separate executables would help. But that’s another rant altogether...

There are differences between bash and sh, and they are actually documented in the man page. Unfortunately, I’m still using a bash version 2 (I’m still on Tiger), so my man page won’t be of much help, but check for any Posix compatibility settings.

There is also a shell option (see built-in command shopt) to enable the true -n behaviour. We are using that at ESO in most shell scripts so that echo behaves the same under Linux and Solaris. – Michael Pruemm

Update: I was wrong about the shell option. What ESO uses at the top of shell scripts is this:

  if [ "`uname`" = "Linux" ]; then enable -n echo; fi

This essentially disables the built-in echo command and forces the scripts to use echo binary. It’s the same solution as yours.

I looked at shopt in my bash’s manpage, but didn’t see anything helpful. The only option having to do with echo was this:

  xpg_echo
     If set, the echo builtin expands backslash-escape sequences by default

That didn’t sound very promising. But I just tried it. Oddly enough, it fixes the problem! If I start bash as sh I get this (this is on the OSX machine):

  [david@monad ~]$ sh
  [david@monad ~]$ shopt xpg_echo
  xpg_echo       	on
  [david@monad ~]$ echo -n hi
  -n hi
  [david@monad ~]$ shopt -u xpg_echo
  [david@monad ~]$ shopt xpg_echo
  xpg_echo       	off
  [david@monad ~]$ echo -n hi
  hi[david@monad ~]$

Doing the same with bash as bash:

  [david@monad ~]$ bash
  [david@monad ~]$ shopt xpg_echo
  xpg_echo       	off
  [david@monad ~]$ echo -n hi
  hi[david@monad ~]$ shopt -s xpg_echo
  [david@monad ~]$ shopt xpg_echo
  xpg_echo       	on
  [david@monad ~]$ echo -n hi
  hi[david@monad ~]$

That’s just goofy.

And I’m not sure this option would even exist in a true /bin/sh, so I’ll stick with my solution.

This is easy to explain, and also in the man page: If invoked as sh, bash tries to be a Posix shell, so -n will not work, instead you have to say: echo “hi\c”. If invoked as bash, it supports the -n option properly.