阿男的小窝

View the Project on GitHub

使用sed删除空白行

本文介绍如何使用sed删除空白行,以及在macos下安装gnu-sed的方法。

创建一个foo.txt

This is a sample file

above is a blank line.


two blank lines above.
   
blank lines with spaces above.

撰写sed脚本删除空白行(包括带空格的空白行):

$ sed '/^\s*$/d' foo.txt
This is a sample file
above is a blank line.
two blank lines above.
blank lines with spaces above.

把处理后的结果输出到新的文件里:

$ sed '/^\s*$/d' foo.txt > foo-with-blank-lines-removed.txt

查看处理后的文件:

$ cat foo-with-blank-lines-removed.txt
This is a sample file
above is a blank line.
two blank lines above.
blank lines with spaces above.

也可以使用sed-ie选项,直接修改原始文件:

$ cat foo.txt
This is a sample file

above is a blank line.


two blank lines above.

blank lines with spaces above.
$ sed -ie '/^\s*$/d' foo.txt

查看直接修改后的文件:

$ cat foo.txt
This is a sample file
above is a blank line.
two blank lines above.
blank lines with spaces above.

注意MacOS下的sed指令不是标准的GNU版本,使用起来会有些不兼容性的问题。最好安装Homebrew提供的sed

$ brew install gnu-sed
...
If you need to use these commands with their normal names, you
can add a "gnubin" directory to your PATH from your bashrc like:
  PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"
...
Additionally, you can access their man pages with normal names if you add
the "gnuman" directory to your MANPATH from your bashrc as well:
  MANPATH="/usr/local/opt/gnu-sed/libexec/gnuman:$MANPATH"

如上安装好后,按照上面的信息提示,把相关路径放进.bash_profile

echo 'export PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"' >> ~/.bash_profile
echo 'export MANPATH="/usr/local/opt/gnu-sed/libexec/gnuman:$MANPATH"' >> ~/.bash_profile

然后加载新的环境文件:

$ . ~/.bash_profile

查看所使用的sed已经是brew安装的版本:

$ which sed
/usr/local/opt/gnu-sed/libexec/gnubin/sed

这样,就可以保证了macos和linux下面所使用的sed的一致性了。

附加,删除一行末尾的空格:

$ sed 's/[[:blank:]]*$//' file

这些末尾的空格被删掉了:

联合起来的命令:

$ sed -ie 's/[[:blank:]]*$//' foo.txt
$ sed -ie '/^\s*$/d' foo.txt

这样就完成了对原始文件的修改。

此外,还可以批量修改一个目录里的所有文件:

for f in $(ls -1)
do
sed -ie 's/[[:blank:]]*$//' $f
sed -ie '/^\s*$/d' $f
done

查看执行结果:

$ set -x
+ set -x
$ for f in $(ls -1)
$ do
$ sed -ie 's/[[:blank:]]*$//' $f
$ sed -ie '/^\s*$/d' $f
$ done
++ ls -v -G -1
+ for f in '$(ls -1)'
+ sed -ie 's/[[:blank:]]*$//' abc.txt
+ sed -ie '/^\s*$/d' abc.txt
+ for f in '$(ls -1)'
+ sed -ie 's/[[:blank:]]*$//' copy.txt
+ sed -ie '/^\s*$/d' copy.txt
+ for f in '$(ls -1)'
+ sed -ie 's/[[:blank:]]*$//' foo.txt
+ sed -ie '/^\s*$/d' foo.txt
+ for f in '$(ls -1)'
+ sed -ie 's/[[:blank:]]*$//' untitled.txt
+ sed -ie '/^\s*$/d' untitled.txt
+ for f in '$(ls -1)'
+ sed -ie 's/[[:blank:]]*$//' untitled.txte
+ sed -ie '/^\s*$/d' untitled.txte

上面的命令里,set -x是打开bash的debug功能,可以看到执行过程。

如果需要处理所有的子目录,就使用ls -R选项:

for f in $(ls -1R)
do
sed -ie 's/[[:blank:]]*$//' $f
sed -ie '/^\s*$/d' $f
done

注意这些命令都会直接修改原始文件,所以要小心操作。