BASH: here-document at line n delimited by end-of-file (wanted `EOF’)

On one of our Debian hosts, we use bash scripts and cron jobs to automate certain tasks. One of these bash scripts downloads files from an FTP server and archives them. After upgrading the host machine to Debian 6.0.4, one of the bash scripts suddenly showed warnings:

/srv/foo/bar.sh: line 146: warning: here-document at line 140 delimited by end-of-file (wanted `EOF')

Whoops, so let’s look into it. The change was probably introduced with the new version of bash:

$ bash --version
GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)

So let’s look into the script itself. The following lines are used to get a directory listing from an FTP server using the ftp command and these are also the lines that throw the warning. The script reads as follows:

FTP_DIR_LISTING=`ftp -n $FTP_SERVER << EOF
        user $FTP_USERNAME $FTP_PASSWORD
        cd $FTP_DIRECTORY
        dir
        bye
        EOF`

It turns out (also see here and here), that the \t in front of the "EOF" is a bit problematic, as is any character after "EOF".

So to solve the problem, simply remove any characters before and after the "EOF". This means to remove all tabs, whitespaces, special characters etc. before and after the EOF, excluding the bash syntax:

FTP_DIR_LISTING=`ftp -n $FTP_SERVER << EOF
        user $FTP_USERNAME $FTP_PASSWORD
        cd $FTP_DIRECTORY
        dir
        bye
EOF`

Now the scripts runs fine and the output of the ftp command is saved in the variable FTP_DIR_LISTING. This output can now be processed with awk. Here is an example:

echo $FTP_DIR_LISTING | awk '{for(i=9;i<=90;i+=9) printf "%s\n",$i};' | while read SINGLE_FILENAME; do
	echo $SINGLE_FILENAME
done

Comments

Oh no, new comments are currently disabled.
If you want to get in touch with me, please do so via e-mail:

rao, on 2015-12-08 13:19:43

Thank you so much,it has very usefull

Bencila, on 2015-09-25 06:26:25

Thanks for posting,it worked for me as well :)

Carmen, on 2015-09-24 12:47:41

Thanks for posting it, it worked for me too.

Jerry, on 2014-09-03 14:35:44

thanks . it's very useful for me

green4s, on 2014-07-24 03:59:19

Very thanks! Very Very Useful Tip!

ROB, on 2014-03-14 08:06:58

Very useful !! thanks a lot

Paul, on 2013-10-30 06:22:03

Legend thankyou helped me too

Alan D'Avila, on 2013-09-20 19:15:12

Wow!!! You just solved my problems! Thank you very much.

Simon Krenger, on 2013-02-03 14:15:20 (Website)

Hello Ivan, thanks for your comment, I updated the post to be even clearer.

Ivan Rosina, on 2013-02-03 12:42:39

EOF must be the only 3chars on the line, check for empty spaces after 'F'

Carlos Núñez, on 2013-01-09 10:45:53

Great advice!. I had this problem and I couldn't figure it out why. Your solution worked like charm. Strange why this happens, specially if you have some indented code and you have to "break" the rule by moving the EOF to the beginning of the line. Simple, but great post!.

Simon Krenger, on 2012-11-11 23:21:17 (Website)

Hello Fabricio, thank you very much for your comment, I added this information to the post!

Fabricio, on 2012-11-11 22:18:04

I found that any character after EOF is not acceptable too.

From_china, on 2012-10-18 11:00:05

hi, thanks for you posting,it's useful to me though just little hints;

Zhuanghui Guo, on 2012-08-30 06:24:14

It's wonderful, thanks for your posting