diff --git a/README.md b/README.md index c117cdad..8ebb5811 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,26 @@ -:snail: useful-scripts [![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html) +🐌 useful-scripts ==================================== +[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html) +[![Join the chat at https://gitter.im/oldratlee/useful-scripts](https://badges.gitter.im/oldratlee/useful-scripts.svg)](https://gitter.im/oldratlee/useful-scripts?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) +[![GitHub release](https://img.shields.io/github/release/oldratlee/useful-scripts.svg)](https://github.com/oldratlee/useful-scripts/releases) [![GitHub stars](https://img.shields.io/github/stars/oldratlee/useful-scripts.svg?style=social&label=Star&)](https://github.com/oldratlee/useful-scripts/stargazers) [![GitHub forks](https://img.shields.io/github/forks/oldratlee/useful-scripts.svg?style=social&label=Fork&)](https://github.com/oldratlee/useful-scripts/fork) +> ❗️ `master`是 不再开发的老版本`1.x`的分支 ❗️ +> +> 请切到 [新版本的开发分支](https://github.com/oldratlee/useful-scripts)。 -把平时有用的手动操作做成脚本，这样可以便捷的使用。 :sparkles: +👉 把平时有用的手动操作做成脚本，这样可以便捷的使用。 ✨ -有自己用的好的脚本 或是 平时常用但没有写成脚本的功能，欢迎提供（[提交Issue](https://github.com/oldratlee/useful-scripts/issues))和分享（[Fork后提交代码](https://github.com/oldratlee/useful-scripts/fork)）！ :sparkling_heart: +有自己用的好的脚本 或是 平时常用但没有写成脚本的功能，欢迎提供（[提交Issue](https://github.com/oldratlee/useful-scripts/issues))和分享（[Fork后提交代码](https://github.com/oldratlee/useful-scripts/fork)）！ 💖 -:beginner: 快速下载&使用 +本仓库的脚本（如`Java`相关脚本）在阿里等公司（如随身云，见[`awesome-scripts`仓库](https://github.com/Suishenyun/awesome-scripts)说明）的线上生产环境部署使用。 +如果你的公司有部署使用，欢迎使用通过[提交Issue](https://github.com/oldratlee/useful-scripts/issues)告知，方便互相交流反馈～ 💘 + +🔰 快速下载&使用 ---------------------- ```bash @@ -20,46 +29,46 @@ source <(curl -fsSL https://raw.githubusercontent.com/oldratlee/useful-scripts/m 更多下载&使用方式，参见[下载使用](docs/install.md)。 -:books: 使用文档 +📚 使用文档 ---------------------- -### :coffee: [`Java`相关脚本](docs/java.md) +### ☕ [`Java`相关脚本](docs/java.md) -1. [show-busy-java-threads.sh](docs/java.md#beer-show-busy-java-threadssh) +1. [show-busy-java-threads](docs/java.md#-show-busy-java-threads) 用于快速排查`Java`的`CPU`性能问题(`top us`值过高)，自动查出运行的`Java`进程中消耗`CPU`多的线程，并打印出其线程栈，从而确定导致性能问题的方法调用。 -1. [show-duplicate-java-classes](docs/java.md#beer-show-duplicate-java-classes) +1. [show-duplicate-java-classes](docs/java.md#-show-duplicate-java-classes) 找出`jar`文件和`class`目录中的重复类。用于排查`Java`类冲突问题。 -1. [find-in-jars.sh](docs/java.md#beer-find-in-jarssh) +1. [find-in-jars](docs/java.md#-find-in-jars) 在目录下所有`jar`文件里，查找类或资源文件。 -### :shell: [`Shell`相关脚本](docs/shell.md) +### 🐚 [`Shell`相关脚本](docs/shell.md) `Shell`使用加强： -1. [c](docs/shell.md#beer-c) +1. [c](docs/shell.md#-c) 原样命令行输出，并拷贝标准输出到系统剪贴板，省去`CTRL+C`操作，优化命令行与其它应用之间的操作流。 -1. [colines](docs/shell.md#beer-colines) +1. [coat](docs/shell.md#-coat) 彩色`cat`出文件行，方便人眼区分不同的行。 -1. [a2l](docs/shell.md#beer-a2l) +1. [a2l](docs/shell.md#-a2l) 按行彩色输出参数，方便人眼查看。 -1. [ap and rp](docs/shell.md#beer-ap-and-rp) +1. [ap and rp](docs/shell.md#-ap-and-rp) 批量转换文件路径为绝对路径/相对路径，会自动跟踪链接并规范化路径。 -1. [tcp-connection-state-counter.sh](docs/shell.md#beer-tcp-connection-state-countersh) +1. [tcp-connection-state-counter](docs/shell.md#-tcp-connection-state-counter) 统计各个`TCP`连接状态的个数。用于方便排查系统连接负荷问题。 -1. [xpl and xpf](docs/shell.md#beer-xpl-and-xpf) +1. [xpl and xpf](docs/shell.md#-xpl-and-xpf) 在命令行中快速完成 在文件浏览器中 打开/选中 指定的文件或文件夹的操作，优化命令行与其它应用之间的操作流。 `Shell`开发/测试加强： -1. [echo-args.sh](docs/shell.md#beer-echo-argssh) +1. [echo-args](docs/shell.md#-echo-args) 输出脚本收到的参数，在控制台运行时，把参数值括起的括号显示成 **红色**，方便人眼查看。用于调试脚本参数输入。 -1. [console-text-color-themes.sh](docs/shell.md#beer-console-text-color-themessh) +1. [console-text-color-themes.sh](docs/shell.md#-console-text-color-themessh) 显示`Terminator`的全部文字彩色组合的效果及其打印方式，用于开发`Shell`的彩色输出。 -1. [parseOpts.sh](docs/shell.md#beer-parseoptssh) +1. [parseOpts.sh](docs/shell.md#-parseoptssh) 命令行选项解析库，加强支持选项有多个值（即数组）。 -### :watch: [`VCS`相关脚本](docs/vcs.md) +### ⌚ [`VCS`相关脚本](docs/vcs.md) -目前`VCS`的脚本都是`svn`分支相关的操作。使用更现代的`Git`吧！ :boom: +目前`VCS`的脚本都是`svn`分支相关的操作。使用更现代的`Git`吧！ 💥 因为不推荐使用`svn`，这里不再列出有哪些脚本了，如果你有兴趣可以点上面链接去看。 diff --git a/a2l b/a2l index 1b9fb8eb..8ecbd5d9 100755 --- a/a2l +++ b/a2l @@ -6,25 +6,25 @@ # $ ./a2l arg1 arg2 # $ ./a2l *.txt # -# @author Jerry Lee +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-a2l +# @author Jerry Lee (oldratlee at gmail dot com) +# NOTE: $'foo' is the escape sequence syntax of bash +readonly ec=$'\033' # escape char +readonly eend=$'\033[0m' # escape end + colorEcho() { local color="$1" shift - if [ -c /dev/stdout ] ; then - # if stdout is console, turn on color output. - echo -ne "\033[1;${color}m" - echo -n "$@" - echo -e "\033[0m" - else - echo "$@" - fi + # check isatty in bash https://stackoverflow.com/questions/10022323 + # if stdout is console, turn on color output. + [ -t 1 ] && echo "$ec[1;${color}m$@$eend" || echo "$@" } -readonly ECHO_COLORS=(37 31 32 34 33 35 56) +readonly -a ECHO_COLORS=(33 35 36 31 32 37 34) COUNT=0 -for a in "$@"; do - colorEcho "${ECHO_COLORS[$((COUNT++ % ${#ECHO_COLORS[@]}))]}" "$a" +for a; do + colorEcho "${ECHO_COLORS[COUNT++ % ${#ECHO_COLORS[@]}]}" "$a" done diff --git a/ap b/ap index 71cf0b20..33ff6fba 100755 --- a/ap +++ b/ap @@ -8,7 +8,8 @@ # # print Absolute Path of arguments. # $ ./ap a.txt ../dir1/b.txt # -# @author Jerry Lee +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-ap-and-rp +# @author Jerry Lee (oldratlee at gmail dot com) [ $# -eq 0 ] && files=(.) || files=("$@") diff --git a/c b/c index 13eb0ac3..f7d6d80c 100755 --- a/c +++ b/c @@ -6,14 +6,85 @@ # $ c echo "hello world!" # $ echo "hello world!" | c # -# @author Jerry Lee +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-c +# @author Jerry Lee (oldratlee at gmail dot com) + +set -e +set -o pipefail + +readonly PROG="`basename "$0"`" + +usage() { + local -r exit_code="$1" + shift + [ -n "$exit_code" -a "$exit_code" != 0 ] && local -r out=/dev/stderr || local -r out=/dev/stdout + + (( $# > 0 )) && { echo "$@"; echo; } > $out + + > $out cat <(content="$(cat)"; echo -n "$content" | copy) + $quiet && local out=/dev/null || local out=/dev/stdout + tee >( + content="$(cat)" + echo $eol "$content" | copy + ) > $out } -if [ $# -eq 0 ]; then +if [ ${#args[@]} -eq 0 ]; then teeAndCopy else - "$@" | teeAndCopy + "${args[@]}" | teeAndCopy fi diff --git a/coat b/coat new file mode 100755 index 00000000..ef3f06d8 --- /dev/null +++ b/coat @@ -0,0 +1,34 @@ +#!/bin/bash +# @Function +# cat lines colorfully. coat means *CO*lorful c*AT*. +# +# @Usage +# $ echo -e 'Hello\nWorld' | coat +# $ coat /path/to/file1 +# $ coat /path/to/file1 /path/to/file2 +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-coat +# @author Jerry Lee (oldratlee at gmail dot com) + +set -e +set -o pipefail + +# if not in console, use cat directly +# check isatty in bash https://stackoverflow.com/questions/10022323 +[ ! -t 1 ] && exec cat "$@" + +# NOTE: $'foo' is the escape sequence syntax of bash +readonly ec=$'\033' # escape char +readonly eend=$'\033[0m' # escape end + +readonly -a ECHO_COLORS=(33 35 36 31 32 37 34) +COUNT=0 +colorEcho() { + local color="${ECHO_COLORS[COUNT++ % ${#ECHO_COLORS[@]}]}" + echo "$ec[1;${color}m$@$eend" +} + +# Bash read line does not read leading spaces https://stackoverflow.com/questions/29689172 +cat "$@" | while IFS= read -r line; do + colorEcho "$line" +done diff --git a/colines b/colines deleted file mode 100755 index 400c6ac3..00000000 --- a/colines +++ /dev/null @@ -1,46 +0,0 @@ -#!/usr/bin/env python -# -*- coding: utf-8 -*- -# @Function -# cat lines colorfully. colines means COLorful LINES. -# -# @Usage -# $ echo -n 'Hello\nWorld' | colines -# $ colines /path/to/file1 -# $ colines /path/to/file1 /path/to/file2 - - -__author__ = 'Jerry Lee' - -import sys - -ECHO_COLORS = (37, 31, 32, 34, 33, 35, 56) -idx = 0 - - -def color_print(*args): - _line = " ".join(args) - global idx - idx += 1 - color = ECHO_COLORS[idx % len(ECHO_COLORS)] - if sys.stdout.isatty(): - print("""\033[1;%sm%s\033[0m""" % (color, _line)) - else: - print(_line) - - -if __name__ == '__main__': - if len(sys.argv) > 1: - for arg in sys.argv[1:]: - if len(sys.argv) > 2: - print('=' * 80) - print(arg) - print('=' * 80) - for line in open(arg).readlines(): - color_print(line.rstrip('\r\n')) - else: - # Read from std input - while True: - line = sys.stdin.readline() - if not line: - break - color_print(line.rstrip('\r\n')) diff --git a/colines b/colines new file mode 120000 index 00000000..98505f2e --- /dev/null +++ b/colines @@ -0,0 +1 @@ +coat \ No newline at end of file diff --git a/console-text-color-themes.sh b/console-text-color-themes.sh index f24bb0f6..7f939e74 100755 --- a/console-text-color-themes.sh +++ b/console-text-color-themes.sh @@ -1,28 +1,28 @@ #!/bin/bash # @Function # show all console text color themes. +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-console-text-color-themessh +# @author Jerry Lee (oldratlee at gmail dot com) readonly _ctct_PROG="$(basename "$(readlink -f "$0")")" [ "$_ctct_PROG" == 'console-text-color-themes.sh' ] && readonly _ctct_is_direct_run=true +readonly _ctct_ec=$'\033' # escape char +readonly _ctct_eend=$'\033[0m' # escape end + colorEcho() { local combination="$1" shift 1 - [ -c /dev/stdout ] && { - echo -e -n "\033[${combination}m" - echo -e -n "$@" - echo -e "\033[0m" - } || echo "$@" + + [ -t 1 ] && echo "$_ctct_ec[${combination}m$@$_ctct_eend" || echo "$@" } colorEchoWithoutNewLine() { local combination="$1" shift 1 - [ -c /dev/stdout ] && { - echo -e -n "\033[${combination}m" - echo -e -n "$@" - echo -e -n "\033[0m" - } || echo -n "$@" + + [ -t 1 ] && echo -n "$_ctct_ec[${combination}m$@$_ctct_eend" || echo -n "$@" } # if not directly run this script(use as lib), just export 2 helper functions, @@ -41,13 +41,22 @@ colorEchoWithoutNewLine() { done echo "Code sample to print color text:" + echo -n ' echo -e "\033[' colorEchoWithoutNewLine "3;35;40" "1;36;41" echo -n "m" colorEchoWithoutNewLine "0;32;40" "Sample Text" echo "\033[0m\"" + + echo -n " echo \$'\033[" + colorEchoWithoutNewLine "3;35;40" "1;36;41" + echo -n "m'\"" + colorEchoWithoutNewLine "0;32;40" "Sample Text" + echo "\"$'\033[0m'" + echo " # NOTE: $'foo' is the escape sequence syntax of bash, safer escape" + echo "Output of above code:" - echo -e " \033[1;36;41mSample Text\033[0m" + echo " $_ctct_ec[1;36;41mSample Text$_ctct_eend" echo echo "If you are going crazy to write text in escapes string like me," echo "you can use colorEcho and colorEchoWithoutNewLine function in this script." diff --git a/cp-svn-url.sh b/cp-svn-url.sh index 320d056f..c8af85ed 100755 --- a/cp-svn-url.sh +++ b/cp-svn-url.sh @@ -6,7 +6,9 @@ # $ ./cp-svn-url.sh # $ ./cp-svn-url.sh /path/to/svn/work/dir # -# @author ivanzhangwb +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/vcs.md#-cp-svn-urlsh +# @author ivanzhangwb (ivanzhangwb at gmail dot com) + readonly PROG=`basename $0` usage() { @@ -25,7 +27,7 @@ EOF exit $1 } -for a in "$@"; do +for a; do [ -h = "$a" -o --help = "$1" ] && usage done @@ -33,21 +35,17 @@ done readonly dir="${1:-.}" -readonly url=$(svn info "${dir}" | awk '/^URL: /{print $2}') +readonly url="$(svn info "${dir}" | awk '/^URL: /{print $2}')" if [ -z "${url}" ]; then echo "Fail to get svn url!" 1>&2 exit 1 fi copy() { - local name=$(uname | tr A-Z a-z) - - case "${name}" in - darwin*) + case "`uname`" in + Darwin*) pbcopy ;; - cygwin*) - clip ;; - mingw*) + CYGWIN*|MINGW*) clip ;; *) xsel -b ;; diff --git a/docs/coat.png b/docs/coat.png new file mode 100644 index 00000000..781cea0b Binary files /dev/null and b/docs/coat.png differ diff --git a/docs/console-colorful-text.png b/docs/console-colorful-text.png index ce0fa92e..a3accacd 100644 Binary files a/docs/console-colorful-text.png and b/docs/console-colorful-text.png differ diff --git a/docs/install.md b/docs/install.md index 78d404a9..d4e2bfa5 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,6 +1,12 @@ -:snail: 下载使用 +🐌 下载使用 ==================================== +> ❗️ `master`是 不再开发的老版本`1.x`的分支 ❗️ +> +> 请切到 [新版本的开发分支](https://github.com/oldratlee/useful-scripts/blob/dev-2.x/docs/install.md)。 + +---------------------- + 下载整个工程的脚本 ------------------- @@ -36,7 +42,7 @@ cd useful-scripts svn up ``` -PS： +PS： 我的做法是把`useful-scripts` checkout到`$HOME/bin/useful-scripts`目录下，再把`$HOME/bin/useful-scripts`配置到`PATH`变量上，这样方便我本地使用所有的脚本。 ### 打包下载 @@ -52,19 +58,19 @@ unzip release.zip 下载和运行单个文件 ------------------- -以[`show-busy-java-threads.sh`](https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads.sh)为例。 +以[`show-busy-java-threads`](https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads)为例。 ### `curl`文件直接用`bash`运行 ```bash -curl -sLk 'https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads.sh' | bash +curl -sLk 'https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads' | bash ``` ### 下载单个文件 ```bash -wget --no-check-certificate https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads.sh -chmod +x show-busy-java-threads.sh +wget --no-check-certificate https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads +chmod +x show-busy-java-threads -./show-busy-java-threads.sh +./show-busy-java-threads ``` diff --git a/docs/java.md b/docs/java.md index f474f128..4754444f 100644 --- a/docs/java.md +++ b/docs/java.md @@ -1,15 +1,21 @@ -:snail: `Java`相关脚本 +🐌 `Java`相关脚本 ==================================== +> ❗️ `master`是 不再开发的老版本`1.x`的分支 ❗️ +> +> 请切到 [新版本的开发分支](https://github.com/oldratlee/useful-scripts/blob/dev-2.x/docs/java.md)。 + +---------------------- + -- [:beer: show-busy-java-threads.sh](#beer-show-busy-java-threadssh) +- [🍺 show-busy-java-threads](#-show-busy-java-threads) - [用法](#%E7%94%A8%E6%B3%95) - [示例](#%E7%A4%BA%E4%BE%8B) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85) -- [:beer: show-duplicate-java-classes](#beer-show-duplicate-java-classes) +- [🍺 show-duplicate-java-classes](#-show-duplicate-java-classes) - [用法](#%E7%94%A8%E6%B3%95-1) - [`JDK`开发场景使用说明](#jdk%E5%BC%80%E5%8F%91%E5%9C%BA%E6%99%AF%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E) - [对于一般的工程](#%E5%AF%B9%E4%BA%8E%E4%B8%80%E8%88%AC%E7%9A%84%E5%B7%A5%E7%A8%8B) @@ -17,56 +23,159 @@ - [`Android`开发场景使用说明](#android%E5%BC%80%E5%8F%91%E5%9C%BA%E6%99%AF%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E) - [示例](#%E7%A4%BA%E4%BE%8B-1) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-1) -- [:beer: find-in-jars.sh](#beer-find-in-jarssh) +- [🍺 find-in-jars](#-find-in-jars) - [用法](#%E7%94%A8%E6%B3%95-2) - [示例](#%E7%A4%BA%E4%BE%8B-2) + - [运行效果](#%E8%BF%90%E8%A1%8C%E6%95%88%E6%9E%9C) - [参考资料](#%E5%8F%82%E8%80%83%E8%B5%84%E6%96%99) -:beer: [show-busy-java-threads.sh](../show-busy-java-threads.sh) +------------------------------- + +关于`Java`排错与诊断，力荐️`Arthas` ❤️ + +- [alibaba/arthas: Alibaba Java诊断利器 - github.com](https://github.com/alibaba/arthas) +- `Arthas`用户文档 https://alibaba.github.io/arthas/ + +`Arthas`功能异常(😜)强劲，且在阿里巴巴线上支持使用多年。我自己也常用，一定要看看用用！ + +`Arthas`是通过`Agent`方式来连接运行的`Java`进程、主要通过交互式来完成功能，与之对应的脚本方式也有其优势，如： + +1. 可以在进程不能启动的情况下完成诊断（如依赖中的重复类分析、`ClassPath`上的资源或类查找） +1. 开销少；简单少依赖（就纯文本的一个脚本文件） +1. 方便与（已有的）工具（如`awk`、`sed`、`cron`）、流程或设施集成，进一步编程/自动化 + +请按需按场景选用。 + +------------------------------- + + + + +🍺 [show-busy-java-threads](../show-busy-java-threads) +---------------------- + +> ❗️ `master`是 不再开发的老版本`1.x`的分支 ❗️ +> +> 请切到 [新版本的开发分支](https://github.com/oldratlee/useful-scripts/blob/dev-2.x/docs/java.md#-show-busy-java-threads)。 + ---------------------- 用于快速排查`Java`的`CPU`性能问题(`top us`值过高)，自动查出运行的`Java`进程中消耗`CPU`多的线程，并打印出其线程栈，从而确定导致性能问题的方法调用。 -目前只支持`Linux`。原因是`Mac`、`Windows`的`ps`命令不支持列出线程线程，更多信息参见[#33](https://github.com/oldratlee/useful-scripts/issues/33)，欢迎提供解法。 +目前只支持`Linux`。原因是`Mac`、`Windows`的`ps`命令不支持列出进程的线程`id`，更多信息参见[#33](https://github.com/oldratlee/useful-scripts/issues/33)，欢迎提供解法。 -PS，如何操作可以参见[@bluedavy](http://weibo.com/bluedavy)的《分布式Java应用》的【5.1.1 cpu消耗分析】一节，说得很详细： +PS，如何操作可以参见[@bluedavy](http://weibo.com/bluedavy)的[《分布式Java应用》](https://book.douban.com/subject/4848587/)的【5.1.1 `CPU`消耗分析】一节，说得很详细： -1. `top`命令找出有问题`Java`进程及线程`id`： - 1. 开启线程显示模式 - 1. 按`CPU`使用率排序 +1. `top`命令找出消耗`CPU`高的`Java`进程及其线程`id`： + 1. 开启线程显示模式（`top -H`，或是打开`top`后按`H`） + 1. 按`CPU`使用率排序（`top`缺省是按`CPU`使用降序，已经合要求；打开`top`后按`P`可以显式指定按`CPU`使用降序） 1. 记下`Java`进程`id`及其`CPU`高的线程`id` -1. 用进程`id`作为参数，`jstack`有问题的`Java`进程 -1. 手动转换线程`id`成十六进制（可以用`printf %x 1234`） -1. 查找十六进制的线程`id`（可以用`grep`） -1. 查看对应的线程栈 +1. 查看消耗`CPU`高的线程栈： + 1. 用进程`id`作为参数，`jstack`出有问题的`Java`进程 + 1. 手动转换线程`id`成十六进制（可以用`printf %x 1234`） + 1. 在`jstack`输出中查找十六进制的线程`id`（可以用`vim`的查找功能`/0x1234`，或是`grep 0x1234 -A 20`） +1. 查看对应的线程栈，分析问题 -查问题时，会要多次这样操作以确定问题，上面过程**太繁琐太慢了**。 +查问题时，会要多次上面的操作以分析确定问题，这个过程**太繁琐太慢了**。 ### 用法 ```bash -show-busy-java-threads.sh -# 从 所有的 Java进程中找出最消耗CPU的线程（缺省5个），打印出其线程栈。 +show-busy-java-threads +# 从所有运行的Java进程中找出最消耗CPU的线程（缺省5个），打印出其线程栈 + +# 缺省会自动从所有的Java进程中找出最消耗CPU的线程，这样用更方便 +# 当然你可以手动指定要分析的Java进程Id，以保证只会显示你关心的那个Java进程的信息 +show-busy-java-threads -p <指定的Java进程Id> + +show-busy-java-threads -c <要显示的线程栈数> -show-busy-java-threads.sh -c <要显示的线程栈数> +show-busy-java-threads <重复执行的间隔秒数> [<重复执行的次数>] +# 多次执行；这2个参数的使用方式类似vmstat命令 -show-busy-java-threads.sh -c <要显示的线程栈数> -p <指定的Java Process> -# -F选项：执行jstack命令时加上-F选项（强制jstack），一般情况不需要使用 -show-busy-java-threads.sh -p <指定的Java Process> -F +show-busy-java-threads -a <运行输出的记录到的文件> +# 记录到文件以方便回溯查看 + +show-busy-java-threads -S <存储jstack输出文件的目录> +# 指定jstack输出文件的存储目录，方便记录以后续分析 ############################## # 注意： ############################## -# 如果Java进程的用户 与 执行脚本的当前用户 不同，则jstack不了这个Java进程。 +# 如果Java进程的用户 与 执行脚本的当前用户 不同，则jstack不了这个Java进程 # 为了能切换到Java进程的用户，需要加sudo来执行，即可以解决： -sudo show-busy-java-threads.sh +sudo show-busy-java-threads + +show-busy-java-threads -s <指定jstack命令的全路径> +# 对于sudo方式的运行，JAVA_HOME环境变量不能传递给root， +# 而root用户往往没有配置JAVA_HOME且不方便配置， +# 显式指定jstack命令的路径就反而显得更方便了 + +# -m选项：执行jstack命令时加上-m选项，显示上Native的栈帧，一般应用排查不需要使用 +show-busy-java-threads -m +# -F选项：执行jstack命令时加上 -F 选项（如果直接jstack无响应时，用于强制jstack），一般情况不需要使用 +show-busy-java-threads -F +# -l选项：执行jstack命令时加上 -l 选项，显示上更多相关锁的信息，一般情况不需要使用 +# 注意：和 -m -F 选项一起使用时，可能会大大增加jstack操作的耗时 +show-busy-java-threads -l + +# 帮助信息 +$ show-busy-java-threads -h +Usage: show-busy-java-threads [OPTION]... [delay [count]] +Find out the highest cpu consumed threads of java processes, +and print the stack of these threads. + +Example: + show-busy-java-threads # show busy java threads info + show-busy-java-threads 1 # update every 1 second, (stop by eg: CTRL+C) + show-busy-java-threads 3 10 # update every 3 seconds, update 10 times + +Output control: + -p, --pid find out the highest cpu consumed threads from + the specified java process. + default from all java process. + -c, --count set the thread count to show, default is 5. + -a, --append-file specifies the file to append output as log. + -S, --store-dir specifies the directory for storing + the intermediate files, and keep files. + default store intermediate files at tmp dir, + and auto remove after run. use this option to keep + files so as to review jstack/top/ps output later. + delay the delay between updates in seconds. + count the number of updates. + delay/count arguments imitates the style of + vmstat command. + +jstack control: + -s, --jstack-path specifies the path of jstack command. + -F, --force set jstack to force a thread dump. use when jstack + does not respond (process is hung). + -m, --mix-native-frames set jstack to print both java and native frames + (mixed mode). + -l, --lock-info set jstack with long listing. + prints additional information about locks. + +CPU usage calculation control: + -d, --top-delay specifies the delay between top samples. + default is 0.5 (second). get thread cpu percentage + during this delay interval. + more info see top -d option. eg: -d 1 (1 second). + -P, --use-ps use ps command to find busy thread(cpu usage) + instead of top command. + default use top command, because cpu usage of + ps command is expressed as the percentage of + time spent running during the *entire lifetime* + of a process, this is not ideal in general. + +Miscellaneous: + -h, --help display this help and exit. ``` ### 示例 ```bash -$ show-busy-java-threads.sh +$ show-busy-java-threads [1] Busy(57.0%) thread(23355/0x5b3b) stack of java process(23269) under user(admin): "pool-1-thread-1" prio=10 tid=0x000000005b5c5000 nid=0x5b3b runnable [0x000000004062c000] java.lang.Thread.State: RUNNABLE @@ -94,7 +203,9 @@ $ show-busy-java-threads.sh at com.xxx.foo.services.common.utils.AliTimer$2.run(AliTimer.java:128) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) -... + at java.lang.Thread.run(Thread.java:662) + +...... ``` 上面的线程栈可以看出，`CPU`消耗最高的2个线程都在执行`java.text.DateFormat.format`，业务代码对应的方法是`shared.monitor.schedule.AppMonitorDataAvgScheduler.run`。可以基本确定： @@ -102,8 +213,9 @@ $ show-busy-java-threads.sh - `AppMonitorDataAvgScheduler.run`调用`DateFormat.format`次数比较频繁。 - `DateFormat.format`比较慢。（这个可以由`DateFormat.format`的实现确定。） -多个执行几次`show-busy-java-threads.sh`，如果上面情况高概率出现，则可以确定上面的判定。 -\# 因为调用越少代码执行越快，则出现在线程栈的概率就越低。 +多执行几次`show-busy-java-threads`，如果上面情况高概率出现，则可以确定上面的判定。 +因为调用越少代码执行越快，则出现在线程栈的概率就越低。 +脚本有自动多次执行的功能，指定 重复执行的间隔秒数/重复执行的次数 参数。 分析`shared.monitor.schedule.AppMonitorDataAvgScheduler.run`实现逻辑和调用方式，以优化实现解决问题。 @@ -113,8 +225,22 @@ $ show-busy-java-threads.sh - [liuyangc3](https://github.com/liuyangc3) - 发现并解决`jstack`非当前用户`Java`进程的问题。 [#50](https://github.com/oldratlee/useful-scripts/pull/50) - 优化性能，通过`read -a`简化反复的`awk`操作。 [#51](https://github.com/oldratlee/useful-scripts/pull/51) +- [superhj1987](https://github.com/superhj1987) / [lirenzuo](https://github.com/lirenzuo) + - 提出/实现了多次执行的功能 [superhj1987/awesome-scripts#1](https://github.com/superhj1987/awesome-scripts/issues/1) +- [xiongchen2012](https://github.com/xiongchen2012) 提出并解决了长用户名截断的Bug [#62](https://github.com/oldratlee/useful-scripts/pull/62) +- [qsLI](https://github.com/qsLI) / [sdslnmd](https://github.com/sdslnmd) + - 发现并提交Issue：show-busy-java-threads支持top来获取cpu占用率，ps的cpu占用率非实时 [#67](https://github.com/oldratlee/useful-scripts/issues/67) +- [geekMessi](https://github.com/geekMessi) + - 发现并提交Issue：在`top v3.2`下提取不正确的Bug [#71](https://github.com/oldratlee/useful-scripts/issues/71) + - 发现并提交Issue：support command name jsvc to find java process [#72](https://github.com/oldratlee/useful-scripts/issues/72) + +🍺 [show-duplicate-java-classes](../show-duplicate-java-classes) +---------------------- + +> ❗️ `master`是 不再开发的老版本`1.x`的分支 ❗️ +> +> 请切到 [新版本的开发分支](https://github.com/oldratlee/useful-scripts/blob/dev-2.x/docs/java.md#-show-duplicate-java-classes)。 -:beer: [show-duplicate-java-classes](../show-duplicate-java-classes) ---------------------- 找出`Java Lib`（`Java`库，即`Jar`文件）或`Class`目录（类目录）中的重复类。 @@ -140,6 +266,15 @@ show-duplicate-java-classes -c path/to/class_dir1 -c /path/to/class_dir2 # 查找指定Class目录和指定目录下所有Jar中的重复类的Jar show-duplicate-java-classes path/to/lib_dir1 /path/to/lib_dir2 -c path/to/class_dir1 -c path/to/class_dir2 + +# 帮助信息 +$ show-duplicate-java-classes -h +Usage: show-duplicate-java-classes [-c class-dir1 [-c class-dir2] ...] [lib-dir1|jar-file1 [lib-dir2|jar-file2] ...] + +Options: + -h, --help show this help message and exit + -c CLASS_DIRS, --class-dir=CLASS_DIRS + add class dir ``` #### `JDK`开发场景使用说明 @@ -263,9 +398,18 @@ class paths to find: ### 贡献者 -[tgic](https://github.com/tg123)提供此脚本。友情贡献者的链接[commandlinefu.cn](http://commandlinefu.cn/)|[微博linux命令行精选](http://weibo.com/u/2674868673) +[tgic](https://github.com/tg123)提供此脚本。友情贡献者的链接 [commandlinefu.cn](http://commandlinefu.cn/) | [微博linux命令行精选](http://weibo.com/u/2674868673) + + + + +🍺 [find-in-jars](../find-in-jars) +---------------------- + +> ❗️ `master`是 不再开发的老版本`1.x`的分支 ❗️ +> +> 请切到 [新版本的开发分支](https://github.com/oldratlee/useful-scripts/blob/dev-2.x/docs/java.md#-find-in-jars)。 -:beer: [find-in-jars.sh](../find-in-jars.sh) ---------------------- 在当前目录下所有`jar`文件里，查找类或资源文件。 @@ -275,47 +419,98 @@ class paths to find: ```bash # 在当前目录下所有`jar`文件里，查找类或资源文件。 -find-in-jars.sh 'log4j\.properties' -find-in-jars.sh 'log4j\.xml$' -find-in-jars.sh log4j\\.xml$ # 和上面命令一样，Shell转义的不同写法而已 -find-in-jars.sh '/(log4j\.properties|log4j\.xml)$' +find-in-jars 'log4j\.properties' +find-in-jars 'log4j\.xml$' +find-in-jars log4j\\.xml$ # 和上面命令一样，Shell转义的不同写法而已 +find-in-jars 'log4j\.(properties|xml)$' # -d选项 指定 查找目录（覆盖缺省的当前目录） -find-in-jars.sh 'log4j\.properties$' -d /path/to/find/directory -# 支持多个查找目录 -find-in-jars.sh 'log4j\.properties' -d /path/to/find/directory1 -d /path/to/find/directory2 +find-in-jars 'log4j\.properties$' -d /path/to/find/directory +# 支持多个查找目录，多次指定这个选项即可 +find-in-jars 'log4j\.properties' -d /path/to/find/directory1 -d /path/to/find/directory2 + +# -e选项 指定 查找`zip`文件的扩展名，缺省是`jar` +find-in-jars 'log4j\.properties' -e zip +# 支持多种查找扩展名，多次指定这个选项即可 +find-in-jars 'log4j\.properties' -e jar -e zip + +# -a选项 指定 查找结果中的Jar文件使用绝对路径 +# 分享给别人时，Jar文件路径是完整的，方便别人找到文件 +find-in-jars 'log4j\.properties' -a + +# -s选项 指定 查找结果中的Jar文件和Jar文件里的查找Entry间分隔符，缺省是『!』 +# 方便你喜欢的人眼查看，或是与工具脚本如`awk`的处理 +find-in-jars 'log4j\.properties' -s ' <-> ' +find-in-jars 'log4j\.properties' -s ' ' | awk '{print $2}' + +# 帮助信息 +$ find-in-jars -h +Usage: find-in-jars [OPTION]... PATTERN +Find files in the jar files under specified directory, +search jar files recursively(include subdirectory). +The pattern default is *extended* regex. + +Example: + find-in-jars 'log4j\.properties' + # search file log4j.properties/log4j.xml at zip root + find-in-jars '^log4j\.(properties|xml)$' + find-in-jars 'log4j\.properties$' -d /path/to/find/directory + find-in-jars '\.properties$' -d /path/to/find/dir1 -d path/to/find/dir2 + find-in-jars 'Service\.class$' -e jar -e zip + find-in-jars 'Mon[^$/]*Service\.class$' -s ' <-> ' + +Find control: + -d, --dir the directory that find jar files. + default is current directory. this option can specify + multiply times to find in multiply directories. + -e, --extension set find file extension, default is jar. this option + can specify multiply times to find multiply extension. + -E, --extended-regexp PATTERN is an extended regular expression (*default*) + -F, --fixed-strings PATTERN is a set of newline-separated strings + -G, --basic-regexp PATTERN is a basic regular expression + -P, --perl-regexp PATTERN is a Perl regular expression + -i, --ignore-case ignore case distinctions + +Output control: + -a, --absolute-path always print absolute path of jar file + -s, --separator specify the separator between jar file and zip entry. + default is `!'. + +Miscellaneous: + -h, --help display this help and exit ``` -注意，后面Pattern是`grep`的 **扩展**正则表达式。 +注意，Pattern缺省是`grep`的 **扩展**正则表达式。 ### 示例 ```bash # 在当前目录下的所有Jar文件中，查找出 log4j.properties文件 -$ find-in-jars.sh 'log4j\.properties$' +$ find-in-jars 'log4j\.properties$' ./hadoop-core-0.20.2-cdh3u3.jar!log4j.properties -# 查找出 以Service结尾的类 -$ ./find-in-jars 'Service.class$' -./WEB-INF/libs/spring-2.5.6.SEC03.jar!org/springframework/stereotype/Service.class -./rpc-benchmark-0.0.1-SNAPSHOT.jar!com/taobao/rpc/benchmark/service/HelloService.class +# 查找出 以Service结尾的类，Jar文件路径输出成绝对路径 +$ find-in-jars 'Service.class$' -a +/home/foo/deploy/app/WEB-INF/libs/spring-2.5.6.SEC03.jar!org/springframework/stereotype/Service.class +/home/foo/deploy/app/WEB-INF/libs/rpc-hello-0.0.1-SNAPSHOT.jar!com/taobao/biz/HelloService.class ...... # 在指定的多个目录的Jar文件中，查找出 properties文件 -$ find-in-jars.sh '\.properties$' -d ../WEB-INF/lib -d ../deploy/lib | grep -v '/pom\.properties$' -../WEB-INF/lib/aspectjtools-1.6.2.jar!org/aspectj/ajdt/ajc/messages.properties -../WEB-INF/lib/aspectjtools-1.6.2.jar!org/aspectj/ajdt/internal/compiler/parser/readableNames.properties -../WEB-INF/lib/aspectjweaver-1.8.8.jar!org/aspectj/weaver/XlintDefault.properties -../WEB-INF/lib/aspectjweaver-1.8.8.jar!org/aspectj/weaver/weaver-messages.properties +$ find-in-jars '\.properties$' -d WEB-INF/lib -d ../deploy/lib | grep -v '/pom\.properties$' +WEB-INF/lib/aspectjtools-1.6.2.jar!org/aspectj/ajdt/ajc/messages.properties +WEB-INF/lib/aspectjweaver-1.8.8.jar!org/aspectj/weaver/XlintDefault.properties ../deploy/lib/groovy-all-1.1-rc-1.jar!groovy/ui/InteractiveShell.properties -../deploy/lib/groovy-all-1.1-rc-1.jar!org/codehaus/groovy/tools/shell/CommandAlias.properties ../deploy/lib/httpcore-4.3.3.jar!org/apache/http/version.properties -../deploy/lib/httpmime-4.2.2.jar!org/apache/http/entity/mime/version.properties -../deploy/lib/javax.servlet-api-3.0.1.jar!javax/servlet/LocalStrings_fr.properties ../deploy/lib/javax.servlet-api-3.0.1.jar!javax/servlet/http/LocalStrings_es.properties ...... ``` +### 运行效果 + +支持彩色输出，文件名中的匹配部分以`grep`的高亮方式显示。 + +![find-in-jar screenshot](https://user-images.githubusercontent.com/1063891/33545067-9eb66072-d8a2-11e7-8a77-d815c0979e5e.gif) + ### 参考资料 [在多个Jar(Zip)文件查找Log4J配置文件的Shell命令行](http://oldratlee.com/458/tech/shell/find-file-in-jar-zip-files.html) diff --git a/docs/script-logo.png b/docs/script-logo.png index 2fed7471..790aeaf6 100644 Binary files a/docs/script-logo.png and b/docs/script-logo.png differ diff --git a/docs/shell.md b/docs/shell.md index 09314b22..3cd19f5a 100644 --- a/docs/shell.md +++ b/docs/shell.md @@ -1,37 +1,46 @@ -:snail: `Shell`相关脚本 +🐌 `Shell`相关脚本 ==================================== +> ❗️ `master`是 不再开发的老版本`1.x`的分支 ❗️ +> +> 请切到 [新版本的开发分支](https://github.com/oldratlee/useful-scripts/blob/dev-2.x/docs/shell.md)。 + +---------------------- + - [`Shell`使用加强](#shell%E4%BD%BF%E7%94%A8%E5%8A%A0%E5%BC%BA) - - [:beer: c](#beer-c) - - [示例](#%E7%A4%BA%E4%BE%8B) + - [🍺 c](#-c) + - [用法/示例](#%E7%94%A8%E6%B3%95%E7%A4%BA%E4%BE%8B) - [参考资料](#%E5%8F%82%E8%80%83%E8%B5%84%E6%96%99) - - [:beer: colines](#beer-colines) + - [🍺 coat](#-coat) + - [示例](#%E7%A4%BA%E4%BE%8B) + - [🍺 a2l](#-a2l) - [示例](#%E7%A4%BA%E4%BE%8B-1) - - [:beer: a2l](#beer-a2l) + - [🍺 ap and rp](#-ap-and-rp) - [示例](#%E7%A4%BA%E4%BE%8B-2) - - [:beer: ap and rp](#beer-ap-and-rp) - - [示例](#%E7%A4%BA%E4%BE%8B-3) - - [:beer: tcp-connection-state-counter.sh](#beer-tcp-connection-state-countersh) + - [🍺 tcp-connection-state-counter](#-tcp-connection-state-counter) - [用法](#%E7%94%A8%E6%B3%95) - - [示例](#%E7%A4%BA%E4%BE%8B-4) + - [示例](#%E7%A4%BA%E4%BE%8B-3) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85) - - [:beer: xpl and xpf](#beer-xpl-and-xpf) + - [🍺 xpl and xpf](#-xpl-and-xpf) - [用法](#%E7%94%A8%E6%B3%95-1) - - [示例](#%E7%A4%BA%E4%BE%8B-5) + - [示例](#%E7%A4%BA%E4%BE%8B-4) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-1) - [`Shell`开发/测试加强](#shell%E5%BC%80%E5%8F%91%E6%B5%8B%E8%AF%95%E5%8A%A0%E5%BC%BA) - - [:beer: echo-args.sh](#beer-echo-argssh) - - [示例](#%E7%A4%BA%E4%BE%8B-6) + - [🍺 echo-args](#-echo-args) + - [示例](#%E7%A4%BA%E4%BE%8B-5) - [使用方式](#%E4%BD%BF%E7%94%A8%E6%96%B9%E5%BC%8F) - - [:beer: console-text-color-themes.sh](#beer-console-text-color-themessh) + - [🍺 console-text-color-themes.sh](#-console-text-color-themessh) + - [用法](#%E7%94%A8%E6%B3%95-2) + - [示例](#%E7%A4%BA%E4%BE%8B-6) + - [运行效果](#%E8%BF%90%E8%A1%8C%E6%95%88%E6%9E%9C) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-2) - [参考资料](#%E5%8F%82%E8%80%83%E8%B5%84%E6%96%99-1) - - [:beer: parseOpts.sh](#beer-parseoptssh) - - [用法](#%E7%94%A8%E6%B3%95-2) + - [🍺 parseOpts.sh](#-parseoptssh) + - [用法](#%E7%94%A8%E6%B3%95-3) - [示例](#%E7%A4%BA%E4%BE%8B-7) - [兼容性](#%E5%85%BC%E5%AE%B9%E6%80%A7) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-3) @@ -41,7 +50,7 @@ `Shell`使用加强 ==================================== -:beer: [c](../c) +🍺 [c](../c) ---------------------- 原样命令行输出，并拷贝标准输出到系统剪贴板，省去`CTRL+C`操作，优化命令行与其它应用之间的操作流。 @@ -51,12 +60,14 @@ 更多说明参见[拷贝复制命令行输出放在系统剪贴板上](http://oldratlee.com/post/2012-12-23/command-output-to-clip)。 -### 示例 +### 用法/示例 有3种使用风格，根据需要或是你的偏好选取。 ```bash +############################################################ # 1. 前缀方式，后面跟上要运行的命令 +############################################################ $ c pwd /Users/jerry $ c echo -e 'a\nb' @@ -64,57 +75,114 @@ a b # 这种使用方式，后面跟的命令不能是别名（alias），对于别名可以用下面的使用方式。 +############################################################ # 2. 后缀方式，管道 +############################################################ $ echo -e 'a\nb' | nl | c 1 a 2 b # gb是oh-my-zsh的别名，列出git的分支，需要后缀的方式的使用。 $ gb | c +############################################################ # 3. 从标准输入读取内容。拷贝文件内容时这种方式最直接。 -$ c < id_rsa.pub -ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAz+ETZEgoLeIiC0rjWewdDs0sbo8c...== a@b.com +############################################################ +$ c < ~/.ssh/id_rsa.pub +ssh-rsa EAAAABIwAAAQEAz+ETZEgoLeIiC0rjWewdDs0sbo8c...== a@b.com + +############################################################ +# -q选项：拷贝但不输出。 +# 当输出内容比较多、又不关心输出内容和命令执行进展时，可以使用这个选项。 +############################################################ +$ c -q < ~/.ssh/id_rsa.pub + +# 帮助信息 +$ c --help +Usage: c [OPTION]... [command [command_args ...]] +Run command and put output to system clipper. +If no command is specified, read from stdin(pipe). + +Example: + c echo "hello world!" + c grep -i 'hello world' menu.h main.c + set | c + c -q < ~/.ssh/id_rsa.pub + +Options: + -k, --keep-eol do not trim new line at end of file + -q, --quiet suppress all normal output, default is false + -h, --help display this help and exit ``` ### 参考资料 -[拷贝复制命令行输出放在系统剪贴板上](http://oldratlee.com/post/2012-12-23/command-output-to-clip)，给出了不同系统可用命令。 +- [拷贝复制命令行输出放在系统剪贴板上](http://oldratlee.com/post/2012-12-23/command-output-to-clip)，给出了不同系统可用命令。 +- 关于文本文件最后的换行，参见[Why should text files end with a newline?](https://stackoverflow.com/questions/729692) -:beer: [colines](../colines) +🍺 [coat](../coat) ---------------------- 彩色`cat`出文件行，方便人眼区分不同的行。 -全系统支持（`Python`实现，安装`Python`即可），如`Linux`、`Mac`、`Windows`。 +支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 + +命令支持选项、功能和使用方式与[`cat`命令](https://linux.die.net/man/1/cat)完全一样（实际上读流操作在实现上全部代理给`cat`命令）。 -命令名`colines`意思是`COLorful LINES`。 +命令名`coat`意思是`COlorful cAT`；当然单词`coat`的意思是外套，彩色输入行就像件漂亮的外套～ 😆 +注：之前命名是`colines`（意思是`COLorful LINES`）。 ### 示例 ```bash -$ echo a | colines -a -$ echo -e 'a\nb' | colines -a -b -$ echo -e 'a\nb' | nl | colines -1 a -2 b -$ colines file1 file2.txt -================================================================================ -file1 -================================================================================ -file1 line1 -file1 line2 -================================================================================ -file2.txt -================================================================================ -file2 line1 -file2 line2 +$ echo Hello world | coat +Hello world +$ echo -e 'Hello\nWorld' | coat +Hello +World +$ echo -e 'Hello\nWorld' | nl | coat + 1 Hello + 2 World +$ coat file1 file2.txt +line1 of file1 +line2 of file1 +... +line1 of file2 +line2 of file2 +... + +# 帮助信息 +# 可以看到本人机器上实现代理的`cat`命令是GNU的实现。 +$ coat --help +Usage: cat [OPTION]... [FILE]... +Concatenate FILE(s) to standard output. + +With no FILE, or when FILE is -, read standard input. + + -A, --show-all equivalent to -vET + -b, --number-nonblank number nonempty output lines, overrides -n + -e equivalent to -vE + -E, --show-ends display $ at end of each line + -n, --number number all output lines + -s, --squeeze-blank suppress repeated empty output lines + -t equivalent to -vT + -T, --show-tabs display TAB characters as ^I + -u (ignored) + -v, --show-nonprinting use ^ and M- notation, except for LFD and TAB + --help display this help and exit + --version output version information and exit + +Examples: + cat f - g Output f's contents, then standard input, then g's contents. + cat Copy standard input to standard output. + +GNU coreutils online help: +Full documentation at: +or available locally via: info '(coreutils) cat invocation' ``` -注：上面显示中，没有彩色，在控制台上运行可以看出彩色效果。 +注：上面示例中，没有彩色；在控制台上运行可以看出彩色效果，如下： +![coat screenshot](../docs/coat.png) -:beer: [a2l](../a2l) +🍺 [a2l](../a2l) ---------------------- 按行彩色输出参数，方便人眼查看。 @@ -133,16 +201,16 @@ B.java # zsh支持 **/* 跨目录glob，可以方便搜索，但是输出内容是空格分隔的不方便查看。 # 把参数按行输出方便查看 或是 grep $ a2l **/*.sh +console-text-color-themes.sh swtrunk.sh -tcp-connection-state-counter.sh test-cases/parseOpts-test.sh test-cases/self-installer.sh ... ``` -注：上面显示中，没有彩色，在控制台上运行可以看出彩色效果。 +注：上面示例中，没有彩色；在控制台上运行可以看出彩色效果，和上面的`coat`命令一样。 -:beer: [ap](../ap) and [rp](../rp) +🍺 [ap](../ap) and [rp](../rp) ---------------------- 批量转换文件路径为绝对路径/相对路径，会自动跟踪链接并规范化路径。 @@ -174,7 +242,10 @@ $ rp /home /etc/../etc /home/admin ../../etc ``` -:beer: [tcp-connection-state-counter.sh](../tcp-connection-state-counter.sh) + + + +🍺 [tcp-connection-state-counter](../tcp-connection-state-counter) ---------------------- 统计各个`TCP`连接状态的个数。 @@ -189,13 +260,13 @@ $ rp /home /etc/../etc /home/admin ### 用法 ```bash -tcp-connection-state-counter.sh +tcp-connection-state-counter ``` ### 示例 ```bash -$ tcp-connection-state-counter.sh +$ tcp-connection-state-counter ESTABLISHED 290 TIME_WAIT 212 SYN_SENT 17 @@ -205,16 +276,16 @@ SYN_SENT 17 [sunuslee](https://github.com/sunuslee)改进此脚本，增加对`MacOS`的支持。 [#56](https://github.com/oldratlee/useful-scripts/pull/56) -:beer: [xpl](../xpl) and [xpf](../xpf) +🍺 [xpl](../xpl) and [xpf](../xpf) ---------------------- 在命令行中快速完成 在文件浏览器中 打开/选中 指定的文件或文件夹的操作，优化命令行与其它应用之间的操作流。 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 -* `xpl`：在文件浏览器中打开指定的文件或文件夹。 -\# `xpl`是`explorer`的缩写。 -* `xpf`: 在文件浏览器中打开指定的文件或文件夹，并选中。 -\# `xpf`是`explorer and select file`的缩写。 +- `xpl`：在文件浏览器中打开指定的文件或文件夹。 + `xpl`是`explorer`的缩写。 +- `xpf`: 在文件浏览器中打开指定的文件或文件夹，并选中。 + `xpf`是`explorer and select file`的缩写。 ### 用法 @@ -247,7 +318,10 @@ xpf /path/to/dir1 /path/to/foo1.txt `Shell`开发/测试加强 ==================================== -:beer: [echo-args.sh](../echo-args.sh) + + + +🍺 [echo-args](../echo-args) ---------------------- 在编写脚本时，常常要确认输入参数是否是期望的：参数个数，参数值（可能包含有人眼不容易发现的空格问题）。 @@ -258,8 +332,8 @@ xpf /path/to/dir1 /path/to/foo1.txt ### 示例 ```bash -$ ./echo-args.sh 1 " 2 foo " "3 3" -0/3: [./echo-args.sh] +$ ./echo-args 1 " 2 foo " "3 3" +0/3: [./echo-args] 1/3: [1] 2/3: [ 2 foo ] 3/3: [3 3] @@ -269,32 +343,39 @@ $ ./echo-args.sh 1 " 2 foo " "3 3" 需要查看某个脚本（实际上也可以是其它的可执行程序）输出参数时，可以这么做： -* 把要查看脚本重命名。 -* 建一个`echo-args.sh`脚本的符号链接到要查看参数的脚本的位置，名字和查看脚本一样。 +- 把要查看脚本重命名。 +- 建一个`echo-args`脚本的符号链接到要查看参数的脚本的位置，名字和查看脚本一样。 这样可以不改其它的程序，查看到输入参数的信息。 -:beer: [console-text-color-themes.sh](../console-text-color-themes.sh) +🍺 [console-text-color-themes.sh](../console-text-color-themes.sh) ---------------------- 显示`Terminator`的全部文字彩色组合的效果及其打印方式。 支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 -脚本中，也给出了`colorEcho`和`colorEchoWithoutNewLine`函数更方便输出彩色文本，用法： +脚本中，也给出了`colorEcho`和`colorEchoWithoutNewLine`函数更方便输出彩色文本 + +### 用法 ```bash colorEcho <颜色样式> <要输出的文本>... colorEchoWithoutNewLine <颜色样式> <要输出的文本>... ``` +### 示例 + ```bash +source console-text-color-themes.sh + # 输出红色文本 colorEcho "0;31;40" "Hello world!" # 输出黄色带下划线的文本 colorEchoWithoutNewLine "4;33;40" "Hello world!" "Hello Hell!" ``` -`console-text-color-themes.sh`的运行效果图如下： +### 运行效果 + ![console-text-color-themes.sh的运行效果图](console-colorful-text.png) ### 贡献者 @@ -305,12 +386,13 @@ colorEchoWithoutNewLine "4;33;40" "Hello world!" "Hello Hell!" - [utensil](https://github.com/utensil)的[在Bash下输出彩色的文本](http://utensil.github.io/tech/2007/09/10/colorful-bash.html)，这是篇很有信息量很钻研的文章！ -:beer: [parseOpts.sh](../parseOpts.sh) +🍺 [parseOpts.sh](../parseOpts.sh) ---------------------- 命令行选项解析库，加强支持选项有多个值（即数组）。 -支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 -\# 自己写一个命令行选项解析函数，是因为[`bash`](http://linux.die.net/man/1/bash)的`buildin`命令[`getopts`](http://linux.die.net/man/1/getopts)和加强版本命令[`getopt`](http://linux.die.net/man/1/getopt)都不支持数组的值。 +支持`Linux`、`Mac`、`Windows`（`cygwin`、`MSSYS`）。 + +自己写一个命令行选项解析函数，是因为[`bash`](http://linux.die.net/man/1/bash)的`buildin`命令[`getopts`](http://linux.die.net/man/1/getopts)和加强版本命令[`getopt`](http://linux.die.net/man/1/getopt)都不支持数组的值。 指定选项的多个值（即数组）的风格模仿[`find`](http://linux.die.net/man/1/find)命令的`-exec`选项： @@ -331,8 +413,8 @@ find file: bar.txt - `-`： 无参数的选项。即有选项则把值设置成`true`。这是 ***缺省*** 的类型。 - `:`： 有参数的选项，值只有一个。 -- `+`： 有多个参数值的选项。值列表要以`;`表示结束。 -注意，`;`是`Bash`的元字符（用于一行中多个命令分隔），所以加上转义写成`\;`（当然也可以按你的喜好写成`";"`或`';'`）。 +- `+`： 有多个参数值的选项。值列表要以`;`表示结束。 + 注意，`;`是`Bash`的元字符（用于一行中多个命令分隔），所以加上转义写成`\;`（当然也可以按你的喜好写成`";"`或`';'`）。 实际要解析的输入参数往往是你的脚本参数，这样`parseOpts`函数调用一般是： @@ -343,10 +425,10 @@ parseOpts "a,a-long|b,b-long:|c,c-long+" "$@" 通过约定的全局变量来获取选项和参数： -* 选项名为`a`，通过全局变量`_OPT_VALUE_a`来获取选项的值。 -* 选项名为`a-long`，通过全局变量`_OPT_VALUE_a_long`来获取选项的值。 -即，把选项名的`-`转`_`，再加上前缀`_OPT_VALUE_`对应的全局变量来获得选项值。 -* 除了选项剩下的参数，通过全局变量`_OPT_ARGS`来获取。 +- 选项名为`a`，通过全局变量`_OPT_VALUE_a`来获取选项的值。 +- 选项名为`a-long`，通过全局变量`_OPT_VALUE_a_long`来获取选项的值。 + 即，把选项名的`-`转`_`，再加上前缀`_OPT_VALUE_`对应的全局变量来获得选项值。 +- 除了选项剩下的参数，通过全局变量`_OPT_ARGS`来获取。 按照惯例，输入参数中如果有`--`表示之后参数中不再有选项，即之后都是参数。 @@ -389,17 +471,17 @@ parseOpts "a,a-long|b,b-long:|c,c-long+" -a -b bv -- --c-long c.sh -p pv -q qv a 这个脚本比较复杂，测试过的环境有： 1. `bash --version` -`GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)` -`uname -a` -`Linux foo-host 2.6.32-41-generic #94-Ubuntu SMP Fri Jul 6 18:00:34 UTC 2012 x86_64 GNU/Linux` + `GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)` + `uname -a` + `Linux foo-host 2.6.32-41-generic #94-Ubuntu SMP Fri Jul 6 18:00:34 UTC 2012 x86_64 GNU/Linux` 1. `bash --version` -`GNU bash, version 3.2.53(1)-release (x86_64-apple-darwin14)` -`uname -a` -`Darwin foo-host 14.0.0 Darwin Kernel Version 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64 x86_64 i386 MacBookPro10,1 Darwin` + `GNU bash, version 3.2.53(1)-release (x86_64-apple-darwin14)` + `uname -a` + `Darwin foo-host 14.0.0 Darwin Kernel Version 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64 x86_64 i386 MacBookPro10,1 Darwin` 1. `bash --version` -`GNU bash, version 3.00.15(1)-release (i386-redhat-linux-gnu)` -`uname -a` -`Linux foo-host 2.6.9-103.ELxenU #1 SMP Wed Mar 14 16:31:15 CST 2012 i686 i686 i386 GNU/Linux` + `GNU bash, version 3.00.15(1)-release (i386-redhat-linux-gnu)` + `uname -a` + `Linux foo-host 2.6.9-103.ELxenU #1 SMP Wed Mar 14 16:31:15 CST 2012 i686 i686 i386 GNU/Linux` ### 贡献者 diff --git a/docs/vcs.md b/docs/vcs.md index cdf58678..178d26ef 100644 --- a/docs/vcs.md +++ b/docs/vcs.md @@ -1,26 +1,32 @@ -:snail: `VCS`相关脚本 +🐌 `VCS`相关脚本 ==================================== +> ❗️ `master`是 不再开发的老版本`1.x`的分支 ❗️ +> +> 请切到 [新版本的开发分支](https://github.com/oldratlee/useful-scripts/blob/dev-2.x/docs/vcs.md)。 + +---------------------- + > 你会发现这些脚本都是`svn`分支相关的操作。 > > 个人在使用`Git`的过程中（7年+），并没有发现有对应脚本的需求（侧面反映出`Git`的优秀）。 > 原因：`Git`的概念模型一等公民支持分支，切换分支是件很简单且统一的事，而`svn`不得不涉及仓库的`URL`（不统一简单）。 > -> 我已经在自己的开发机上卸载了`svn`，没有需求场景也没理由再用了。 :stuck_out_tongue: +> 我已经在自己的开发机上卸载了`svn`，没有需求场景也没理由再用了。 😛 > -> 使用更现代的`Git`吧！ :boom: +> 使用更现代的`Git`吧！ 💥 -1. [swtrunk.sh](docs/vcs.md#beer-swtrunksh) +1. [swtrunk.sh](#-swtrunksh) 自动`svn`工作目录从分支（`branches`）切换到主干（`trunk`）。 PS： `Git`对应的是`git checkout master`，如果你使用了`oh-my-zsh`，已经有对应的别名加速了：`gcm`。 -1. [svn-merge-stop-on-copy.sh](docs/vcs.md#beer-svn-merge-stop-on-copysh) +1. [svn-merge-stop-on-copy.sh](#-svn-merge-stop-on-copysh) 把指定的远程分支从刚新建分支以来的修改合并到本地`svn`目录或是另一个远程分支。 PS：`Git`的合并很直接简单，`git merge branch-foo`，也更智能（没有树冲突一说）。 -1. [cp-svn-url.sh](docs/vcs.md#beer-cp-svn-urlsh) +1. [cp-svn-url.sh](#-cp-svn-urlsh) 拷贝当前`svn`目录对应的远程分支到系统的粘贴板，省去`CTRL+C`操作。 PS：`Git`分支不需要`URL`来引用，没有这个脚本的需求，直接给个分支名就好了。 -:beer: [swtrunk.sh](../swtrunk.sh) +🍺 [swtrunk.sh](../swtrunk.sh) ---------------------- `svn`工作目录从分支（`branches`）切换到主干（`trunk`）。 @@ -29,8 +35,9 @@ 命令以`svn`的标准目录命名约定来识别分支和主干。 即，分支在目录`branches`下，主干在目录`trunk`下。 示例： -- 分支： http://www.foo.com/project1/branches/feature1 -- 主干： http://www.foo.com/project1/trunk + +- 分支： +- 主干： ### 用法 @@ -58,7 +65,7 @@ svn work dir /path/to/svn/work/dir1 switch from http://www.foo.com/project1/bran svn work dir /path/to/svn/work/dir2 switch from http://www.foo.com/project2/branches/feature1 to http://www.foo.com/project2/trunk ! ``` -:beer: [svn-merge-stop-on-copy.sh](../svn-merge-stop-on-copy.sh) +🍺 [svn-merge-stop-on-copy.sh](../svn-merge-stop-on-copy.sh) ---------------------- 把指定的远程分支从刚新建分支以来的修改合并到本地`svn`目录或是另一个远程分支。 @@ -84,7 +91,7 @@ svn-merge-stop-on-copy.sh http://www.foo.com/project1/branches/feature1 http://w [姜太公](https://github.com/jzwlqx)提供此脚本。 -:beer: [cp-svn-url.sh](../cp-svn-url.sh) +🍺 [cp-svn-url.sh](../cp-svn-url.sh) ---------------------- 拷贝当前`svn`目录对应的远程分支到系统的粘贴板，省去`CTRL+C`操作。 diff --git a/echo-args b/echo-args new file mode 100755 index 00000000..7562b7c6 --- /dev/null +++ b/echo-args @@ -0,0 +1,27 @@ +#!/bin/bash +# @Function +# print arguments in human and debugging friendly style. +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-echo-args +# @author Jerry Lee (oldratlee at gmail dot com) + +readonly ec=$'\033' # escape char +readonly eend=$'\033[0m' # escape end + +echoArg() { + local index=$1 + local count=$2 + local value=$3 + + # if stdout is console, turn on color output. + [ -t 1 ] && + echo "$index/$count: $ec[1;31m[$eend$ec[0;34;42m$value$eend$ec[1;31m]$eend" || + echo "$index/$count: [$value]" +} + + +echoArg 0 $# "$0" +idx=1 +for a ; do + echoArg $((idx++)) $# "$a" +done diff --git a/echo-args.sh b/echo-args.sh deleted file mode 100755 index dc494c73..00000000 --- a/echo-args.sh +++ /dev/null @@ -1,31 +0,0 @@ -#!/bin/bash - -redEcho() { - if [ -c /dev/stdout ] ; then - # if stdout is console, turn on color output. - echo -ne "\033[1;31m" - echo -n "$@" - echo -ne "\033[0m" - else - echo -n "$@" - fi -} - -echoArg() { - local index=$1 - local count=$2 - local value=$3 - - echo -n "$index/$count: " - redEcho "[" - echo -n "$value" - redEcho "]" - echo -} - - -echoArg 0 $# "$0" -idx=1 -for a ; do - echoArg $((idx++)) $# "$a" -done diff --git a/echo-args.sh b/echo-args.sh new file mode 120000 index 00000000..32c33811 --- /dev/null +++ b/echo-args.sh @@ -0,0 +1 @@ +echo-args \ No newline at end of file diff --git a/find-in-jars b/find-in-jars new file mode 100755 index 00000000..0571a804 --- /dev/null +++ b/find-in-jars @@ -0,0 +1,255 @@ +#!/bin/bash +# @Function +# Find files in the jar files under specified directory, search jar files recursively(include subdirectory). +# +# @Usage +# $ find-in-jars 'log4j\.properties' +# # search file log4j.properties/log4j.xml at zip root +# $ find-in-jars '^log4j\.(properties|xml)$' +# $ find-in-jars 'log4j\.properties$' -d /path/to/find/directory +# $ find-in-jars '\.properties$' -d /path/to/find/dir1 -d path/to/find/dir2 +# $ find-in-jars 'Service\.class$' -e jar -e zip +# $ find-in-jars 'Mon[^$/]*Service\.class$' -s ' <-> ' +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#-find-in-jars +# @author Jerry Lee (oldratlee at gmail dot com) + +readonly PROG="`basename "$0"`" + +################################################################################ +# util functions +################################################################################ + +[ -t 1 ] && readonly is_console=true || readonly is_console=false + +# NOTE: $'foo' is the escape sequence syntax of bash +readonly ec=$'\033' # escape char +readonly eend=$'\033[0m' # escape end +readonly cr=$'\r' # carriage return + +redEcho() { + $is_console && echo "$ec[1;31m$@$eend" || echo "$@" +} + +die() { + redEcho "Error: $@" 1>&2 + exit 1 +} + +# Getting console width using a bash script +# https://unix.stackexchange.com/questions/299067 +$is_console && readonly columns=$(stty size | awk '{print $2}') + +printResponsiveMessage() { + $is_console || return + + local message="$*" + # http://www.linuxforums.org/forum/red-hat-fedora-linux/142825-how-truncate-string-bash-script.html + echo -n "${message:0:columns}" +} + +clearResponsiveMessage() { + $is_console || return + + # How to delete line with echo? + # https://unix.stackexchange.com/questions/26576 + # + # terminal escapes: http://ascii-table.com/ansi-escape-sequences.php + # In particular, to clear from the cursor position to the beginning of the line: + # echo -e "\033[1K" + # Or everything on the line, regardless of cursor position: + # echo -e "\033[2K" + echo -n "$ec[2K$cr" +} + +usage() { + local -r exit_code="$1" + shift + [ -n "$exit_code" -a "$exit_code" != 0 ] && local -r out=/dev/stderr || local -r out=/dev/stdout + + (( $# > 0 )) && { echo "$@"; echo; } > $out + + > $out cat < ' + +Find control: + -d, --dir the directory that find jar files. + default is current directory. this option can specify + multiply times to find in multiply directories. + -e, --extension set find file extension, default is jar. this option + can specify multiply times to find multiply extension. + -E, --extended-regexp PATTERN is an extended regular expression (*default*) + -F, --fixed-strings PATTERN is a set of newline-separated strings + -G, --basic-regexp PATTERN is a basic regular expression + -P, --perl-regexp PATTERN is a Perl regular expression + -i, --ignore-case ignore case distinctions + +Output control: + -a, --absolute-path always print absolute path of jar file + -s, --separator specify the separator between jar file and zip entry. + default is \`!'. + +Miscellaneous: + -h, --help display this help and exit +EOF + + exit $1 +} + +################################################################################ +# parse options +################################################################################ + +declare -a args=() +declare -a dirs=() +while (( $# > 0 )); do + case "$1" in + -d|--dir) + dirs=("${dirs[@]}" "$2") + shift 2 + ;; + -e|--extension) + extension=("${extension[@]}" "$2") + shift 2 + ;; + # support the typo option --seperator for compatibility + -s|--separator|--seperator) + separator="$2" + shift 2 + ;; + -E|--extended-regexp) + regex_mode=-E + shift + ;; + -F|--fixed-strings) + regex_mode=-F + shift + ;; + -G|--basic-regexp) + regex_mode=-G + shift + ;; + -P|--perl-regexp) + regex_mode=-P + shift + ;; + -i|--ignore-case) + ignore_case_option=-i + shift + ;; + -a|--absolute-path) + use_absolute_path=true + shift + ;; + -h|--help) + usage + ;; + --) + shift + args=("${args[@]}" "$@") + break + ;; + -*) + usage 2 "${PROG}: unrecognized option '$1'" + ;; + *) + args=("${args[@]}" "$1") + shift + ;; + esac +done + +dirs=${dirs:-.} +extension=${extension:-jar} +regex_mode=${regex_mode:--E} + +use_absolute_path=${use_absolute_path:-false} +separator="${separator:-!}" + +(( "${#args[@]}" == 0 )) && usage 1 "No find file pattern!" +(( "${#args[@]}" > 1 )) && usage 1 "More than 1 file pattern: ${args[@]}" +readonly pattern="${args[0]}" + +declare -a tmp_dirs=() +for d in "${dirs[@]}"; do + [ -e "$d" ] || die "file $d(specified by option -d) does not exist!" + [ -d "$d" ] || die "file $d(specified by option -d) exists but is not a directory!" + [ -r "$d" ] || die "directory $d(specified by option -d) exists but is not readable!" + # convert dirs to Absolute Path + $use_absolute_path && tmp_dirs=( "${tmp_dirs[@]}" "$(cd "$d" && pwd)" ) +done +# set dirs to Absolute Path +$use_absolute_path && dirs=( "${tmp_dirs[@]}" ) + +# convert extensions to find -iname options +for e in "${extension[@]}"; do + (( "${#find_iname_options[@]}" == 0 )) && + find_iname_options=( -iname "*.$e" ) || + find_iname_options=( "${find_iname_options[@]}" -o -iname "*.$e" ) +done + +################################################################################ +# Check the existence of command for listing zip entry! +################################################################################ + +# `zipinfo -1`/`unzip -Z1` is ~25 times faster than `jar tf`, find zipinfo/unzip command first. +# +# How to list files in a zip without extra information in command line +# https://unix.stackexchange.com/a/128304/136953 +if which zipinfo &> /dev/null; then + readonly command_for_list_zip='zipinfo -1' +elif which unzip &> /dev/null; then + readonly command_for_list_zip='unzip -Z1' +else + if ! which jar &> /dev/null; then + [ -n "$JAVA_HOME" ] || die "jar not found on PATH and JAVA_HOME env var is blank!" + [ -f "$JAVA_HOME/bin/jar" ] || die "jar not found on PATH and \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) file does NOT exists!" + [ -x "$JAVA_HOME/bin/jar" ] || die "jar not found on PATH and \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) is NOT executalbe!" + export PATH="$JAVA_HOME/bin:$PATH" + fi + readonly command_for_list_zip='jar tf' +fi + +################################################################################ +# find logic +################################################################################ + +readonly jar_files="$(find "${dirs[@]}" "${find_iname_options[@]}" -type f)" +readonly total_count="$(echo $(echo "$jar_files" | wc -l))" +[ -n "$jar_files" ] || die "No ${extension[@]} file found!" + +findInJarFiles() { + $is_console && local -r grep_color_option='--color=always' + + local counter=1 + local jar_file + while read jar_file; do + printResponsiveMessage "finding in jar($((counter++))/$total_count): $jar_file" + + $command_for_list_zip "${jar_file}" | + grep $regex_mode $ignore_case_option $grep_color_option -- "$pattern" | + while read file; do + clearResponsiveMessage + + $is_console && + echo "$ec[1;35m${jar_file}${eend}${ec}[1;32m${separator}${eend}${file}" || + echo "${jar_file}${separator}${file}" + done + + clearResponsiveMessage + done +} + +echo "$jar_files" | findInJarFiles diff --git a/find-in-jars.sh b/find-in-jars.sh deleted file mode 100755 index 96ef4c5e..00000000 --- a/find-in-jars.sh +++ /dev/null @@ -1,142 +0,0 @@ -#!/bin/bash -# @Function -# Find file in the jar files under current directory -# -# @Usage -# $ find-in-jars.sh log4j\\.xml -# $ find-in-jars.sh 'log4j\.properties' -# $ find-in-jars.sh 'log4j\.properties|log4j\.xml' -# -# @author Jerry Lee - -readonly PROG=`basename $0` - -usage() { - local out - [ -n "$1" -a "$1" != 0 ] && out=/dev/stderr || out=/dev/stdout - - > $out cat <&2 ; usage 1; } -[ "${#args[@]}" -gt 1 ] && { echo "More than 1 file pattern!" 1>&2 ; usage 1; } -readonly pattern="${args[0]}" - -################################################################################ -# Check the existence of command for listing zip entry! -################################################################################ - -# `zipinfo -1`/`unzip -Z1` is ~25 times faster than `jar tf`, find zipinfo/unzip command first. -# -# How to list files in a zip without extra information in command line -# https://unix.stackexchange.com/a/128304/136953 -if which zipinfo &> /dev/null; then - readonly command_for_list_zip='zipinfo -1' -elif which unzip &> /dev/null; then - readonly command_for_list_zip='unzip -Z1' -else - if ! which jar &> /dev/null; then - [ -z "$JAVA_HOME" ] && { - echo "Error: jar not found on PATH!" 1>&2 - exit 1 - } - ! [ -f "$JAVA_HOME/bin/jar" ] && { - echo "Error: jar not found on PATH and \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) file does NOT exists!" 1>&2 - exit 1 - } - ! [ -x "$JAVA_HOME/bin/jar" ] && { - echo "Error: jar not found on PATH and \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) is NOT executalbe!" 1>&2 - exit 1 - } - export PATH="$JAVA_HOME/bin:$PATH" - fi - - readonly command_for_list_zip='jar tf' -fi - -################################################################################ -# find logic -################################################################################ - -[ -c /dev/stdout ] && readonly is_console=true || readonly is_console=false - -# Getting console width using a bash script -# https://unix.stackexchange.com/questions/299067 -$is_console && readonly columns=$(stty size | awk '{print $2}') - -print_responsive_message() { - $is_console || return - - local message="$*" - # http://www.linuxforums.org/forum/red-hat-fedora-linux/142825-how-truncate-string-bash-script.html - echo -n "${message:0:columns}" -} - -clear_responsive_message() { - $is_console || return - - # How to delete line with echo? - # https://unix.stackexchange.com/questions/26576 - # - # terminal escapes: http://ascii-table.com/ansi-escape-sequences.php - # In particular, to clear from the cursor position to the beginning of the line: - # echo -e "\033[1K" - # Or everything on the line, regardless of cursor position: - # echo -e "\033[2K" - echo -n -e "\033[2K\\r" -} - - -readonly jar_files="$(find "${dirs[@]}" -iname '*.jar')" -readonly total_count="$(echo "$jar_files" | wc -l)" - -counter=1 -while read jar_file; do - print_responsive_message "finding in jar($((counter++))/$total_count): $jar_file" - - $command_for_list_zip "${jar_file}" | grep -E "$pattern" | while read file; do - clear_responsive_message - - echo "${jar_file}"\!"${file}" - done - - clear_responsive_message - -done < <(echo "$jar_files") diff --git a/find-in-jars.sh b/find-in-jars.sh new file mode 120000 index 00000000..cd6c3a87 --- /dev/null +++ b/find-in-jars.sh @@ -0,0 +1 @@ +find-in-jars \ No newline at end of file diff --git a/monitor-host.sh b/monitor-host.sh deleted file mode 100755 index ae91bddb..00000000 --- a/monitor-host.sh +++ /dev/null @@ -1,40 +0,0 @@ -#!/bin/bash -# @Function -# monitor host:network io memory cpu -# -# @Usage -# $ ./monitor-host.sh -# -# @author Bryant Hang - -DELAY=10 -COUNT=60 - -LOG_PATH='monitor_logs' - -if ! [ -d $LOG_PATH ]; then - mkdir -p $LOG_PATH -fi - -cur_date="`date +%Y%m%d`" - -top_log_path=$LOG_PATH'/top_'$cur_date'.log' -memory_log_path=$LOG_PATH'/memory_'$cur_date'.log' -cpu_log_path=$LOG_PATH'/cpu_'$cur_date'.log' -io_log_path=$LOG_PATH'/io_'$cur_date'.log' -network_log_path=$LOG_PATH'/network_'$cur_date'.log' - -# total performance check -top -b -d $DELAY -n $COUNT >> $top_log_path 2>&1 & - -# memory check -vmstat $DELAY $COUNT >> $memory_log_path 2>&1 & - -# cpu check -sar -u $DELAY $COUNT >> $cpu_log_path 2>&1 & - -# IO check -iostat $DELAY $COUNT >> $io_log_path 2>&1 & - -# network check -sar -n DEV $DELAY $COUNT >> $network_log_path 2>&1 & diff --git a/parseOpts.sh b/parseOpts.sh index 9d03e1a9..eb6270aa 100755 --- a/parseOpts.sh +++ b/parseOpts.sh @@ -15,23 +15,22 @@ # _OPT_VALUE_c_long = (c.sh -p pv -q qv arg1) # Array type # _OPT_ARGS = (aa bb cc) # Array type # -# @author Jerry Lee +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-parseoptssh +# @author Jerry Lee (oldratlee at gmail dot com) ##################################################################### -# Utils Methods +# Util Funtions ##################################################################### +# NOTE: $'foo' is the escape sequence syntax of bash +readonly _opts_ec=$'\033' # escape char +readonly _opts_eend=$'\033[0m' # escape end + _opts_colorEcho() { local color=$1 shift - if [ -c /dev/stdout ] ; then - # if stdout is console, turn on color output. - echo -ne "\033[1;${color}m" - echo -n "$@" - echo -e "\033[0m" - else - echo "$@" - fi + # if stdout is console, turn on color output. + [ -t 1 ] && echo "$_opts_ec[1;${color}m$@$_opts_eend" || echo "$@" } _opts_redEcho() { @@ -47,7 +46,7 @@ _opts_convertToVarName() { } ##################################################################### -# Parse Methods +# Parse Functions # # Use Globle Variable: # * _OPT_INFO_LIST_INDEX : Option info, data structure. @@ -312,7 +311,7 @@ parseOpts() { } ##################################################################### -# Show parsed Option Info Methods +# Show parsed Option Info Functions ##################################################################### _opts_showOptDescInfoList() { diff --git a/rp b/rp index 6065063d..d4b0739d 100755 --- a/rp +++ b/rp @@ -8,7 +8,8 @@ # # if more than 1 argument, print relative path to last argument. # $ ./rp a.txt ../b.txt /etc/passwd /etc/apache2 # -# @author Jerry Lee +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-ap-and-rp +# @author Jerry Lee (oldratlee at gmail dot com) [ $# -eq 0 ] && { echo "ERROR: NO argument!" @@ -23,8 +24,8 @@ argc=$# # Get last argument - relativeTo="${argv[$((argc - 1))]}" - files=( "${argv[@]:0:$((argc - 1))}" ) + relativeTo="${argv[argc - 1]}" + files=( "${argv[@]:0:argc - 1}" ) } [ -f "$relativeTo" ] && relativeTo="$(dirname "$relativeTo")" diff --git a/show-busy-java-threads b/show-busy-java-threads new file mode 100755 index 00000000..02e91520 --- /dev/null +++ b/show-busy-java-threads @@ -0,0 +1,461 @@ +#!/bin/bash +# @Function +# Find out the highest cpu consumed threads of java processes, and print the stack of these threads. +# +# @Usage +# $ ./show-busy-java-threads +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#-show-busy-java-threads +# @author Jerry Lee (oldratlee at gmail dot com) +# @author superhj1987 (superhj1987 at 126 dot com) + +readonly PROG="`basename $0`" +readonly -a COMMAND_LINE=("$0" "$@") +# Get current user name via whoami command +# See https://www.lifewire.com/current-linux-user-whoami-command-3867579 +# Because if run command by `sudo -u`, env var $USER is not rewritten/correct, just inherited from outside! +readonly USER="`whoami`" + +################################################################################ +# util functions +################################################################################ + +# NOTE: $'foo' is the escape sequence syntax of bash +readonly ec=$'\033' # escape char +readonly eend=$'\033[0m' # escape end + +colorEcho() { + local color=$1 + shift + + # if stdout is console, turn on color output. + [ -t 1 ] && echo "$ec[1;${color}m$@$eend" || echo "$@" +} + +colorPrint() { + local color=$1 + shift + + colorEcho "$color" "$@" + [ -n "$append_file" -a -w "$append_file" ] && echo "$@" >> "$append_file" + [ -n "$store_dir" -a -w "$store_dir" ] && echo "$@" >> "${store_file_prefix}$PROG" +} + +normalPrint() { + echo "$@" + [ -n "$append_file" -a -w "$append_file" ] && echo "$@" >> "$append_file" + [ -n "$store_dir" -a -w "$store_dir" ] && echo "$@" >> "${store_file_prefix}$PROG" +} + +redPrint() { + colorPrint 31 "$@" +} + +greenPrint() { + colorPrint 32 "$@" +} + +yellowPrint() { + colorPrint 33 "$@" +} + +bluePrint() { + colorPrint 36 "$@" +} + +die() { + redPrint "Error: $@" 1>&2 + exit 1 +} + +logAndRun() { + echo "$@" + echo + "$@" +} + +logAndCat() { + echo "$@" + echo + cat +} + +usage() { + local -r exit_code="$1" + shift + [ -n "$exit_code" -a "$exit_code" != 0 ] && local -r out=/dev/stderr || local -r out=/dev/stdout + + (( $# > 0 )) && { echo "$@"; echo; } > $out + + > $out cat < find out the highest cpu consumed threads from + the specified java process. + default from all java process. + -c, --count set the thread count to show, default is 5. + -a, --append-file specifies the file to append output as log. + -S, --store-dir specifies the directory for storing + the intermediate files, and keep files. + default store intermediate files at tmp dir, + and auto remove after run. use this option to keep + files so as to review jstack/top/ps output later. + delay the delay between updates in seconds. + count the number of updates. + delay/count arguments imitates the style of + vmstat command. + +jstack control: + -s, --jstack-path specifies the path of jstack command. + -F, --force set jstack to force a thread dump. use when jstack + does not respond (process is hung). + -m, --mix-native-frames set jstack to print both java and native frames + (mixed mode). + -l, --lock-info set jstack with long listing. + prints additional information about locks. + +CPU usage calculation control: + -d, --top-delay specifies the delay between top samples. + default is 0.5 (second). get thread cpu percentage + during this delay interval. + more info see top -d option. eg: -d 1 (1 second). + -P, --use-ps use ps command to find busy thread(cpu usage) + instead of top command. + default use top command, because cpu usage of + ps command is expressed as the percentage of + time spent running during the *entire lifetime* + of a process, this is not ideal in general. + +Miscellaneous: + -h, --help display this help and exit. +EOF + + exit $exit_code +} + +################################################################################ +# Check os support +################################################################################ + +uname | grep '^Linux' -q || die "$PROG only support Linux, not support `uname` yet!" + +################################################################################ +# parse options +################################################################################ + +# NOTE: ARGS can not be declared as readonly!! +# readonly declaration make exit code of assignment to be always 0, aka. the exit code of `getopt` in subshell is discarded. +# tested on bash 4.2.46 +ARGS=`getopt -n "$PROG" -a -o p:c:a:s:S:Pd:Fmlh -l count:,pid:,append-file:,jstack-path:,store-dir:,use-ps,top-delay:,force,mix-native-frames,lock-info,help -- "$@"` +[ $? -ne 0 ] && { echo; usage 1; } +eval set -- "${ARGS}" + +while true; do + case "$1" in + -c|--count) + count="$2" + shift 2 + ;; + -p|--pid) + pid="$2" + shift 2 + ;; + -a|--append-file) + append_file="$2" + shift 2 + ;; + -s|--jstack-path) + jstack_path="$2" + shift 2 + ;; + -S|--store-dir) + store_dir="$2" + shift 2 + ;; + -P|--use-ps) + use_ps=true + shift + ;; + -d|--top-delay) + top_delay="$2" + shift 2 + ;; + -F|--force) + force=-F + shift + ;; + -m|--mix-native-frames) + mix_native_frames=-m + shift + ;; + -l|--lock-info) + more_lock_info=-l + shift + ;; + -h|--help) + usage + ;; + --) + shift + break + ;; + esac +done + +count=${count:-5} + +update_delay=${1:-0} +[ -z "$1" ] && update_count=1 || update_count=${2:-0} +(( update_count < 0 )) && update_count=0 + +top_delay=${top_delay:-0.5} +use_ps=${use_ps:-false} + +# check the directory of append-file(-a) mode, create if not exsit. +if [ -n "$append_file" ]; then + if [ -e "$append_file" ]; then + [ -f "$append_file" ] || die "$append_file(specified by option -a, for storing run output files) exists but is not a file!" + [ -w "$append_file" ] || die "file $append_file(specified by option -a, for storing run output files) exists but is not writable!" + else + append_file_dir="$(dirname "$append_file")" + if [ -e "$append_file_dir" ]; then + [ -d "$append_file_dir" ] || die "directory $append_file_dir(specified by option -a, for storing run output files) exists but is not a directory!" + [ -w "$append_file_dir" ] || die "directory $append_file_dir(specified by option -a, for storing run output files) exists but is not writable!" + else + mkdir -p "$append_file_dir" || die "fail to create directory $append_file_dir(specified by option -a, for storing run output files)!" + fi + fi +fi + +# check store directory(-S) mode, create directory if not exsit. +if [ -n "$store_dir" ]; then + if [ -e "$store_dir" ]; then + [ -d "$store_dir" ] || die "$store_dir(specified by option -S, for storing output files) exists but is not a directory!" + [ -w "$store_dir" ] || die "directory $store_dir(specified by option -S, for storing output files) exists but is not writable!" + else + mkdir -p "$store_dir" || die "fail to create directory $store_dir(specified by option -S, for storing output files)!" + fi +fi + +################################################################################ +# check the existence of jstack command +################################################################################ + +if [ -n "$jstack_path" ]; then + [ -f "$jstack_path" ] || die "$jstack_path is NOT found!" + [ -x "$jstack_path" ] || die "$jstack_path is NOT executalbe!" +elif which jstack &> /dev/null; then + jstack_path="`which jstack`" +else + [ -n "$JAVA_HOME" ] || die "jstack not found on PATH and No JAVA_HOME setting! Use -s option set jstack path manually." + [ -f "$JAVA_HOME/bin/jstack" ] || die "jstack not found on PATH and \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) file does NOT exists! Use -s option set jstack path manually." + [ -x "$JAVA_HOME/bin/jstack" ] || die "jstack not found on PATH and \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) is NOT executalbe! Use -s option set jstack path manually." + jstack_path="$JAVA_HOME/bin/jstack" +fi + +################################################################################ +# biz logic +################################################################################ + +readonly run_timestamp="`date "+%Y-%m-%d_%H:%M:%S.%N"`" +readonly uuid="${PROG}_${run_timestamp}_${RANDOM}_$$" + +readonly tmp_store_dir="/tmp/${uuid}" +if [ -n "$store_dir" ]; then + readonly store_file_prefix="$store_dir/${run_timestamp}_" +else + readonly store_file_prefix="$tmp_store_dir/${run_timestamp}_" +fi +mkdir -p "$tmp_store_dir" + +cleanupWhenExit() { + rm -rf "$tmp_store_dir" &> /dev/null +} +trap "cleanupWhenExit" EXIT + +headInfo() { + colorEcho "0;34;42" ================================================================================ + echo "$(date "+%Y-%m-%d %H:%M:%S.%N") [$(( i + 1 ))/$update_count]: ${COMMAND_LINE[@]}" + colorEcho "0;34;42" ================================================================================ + echo +} + +if [ -n "${pid}" ]; then + readonly ps_process_select_options="-p $pid" +else + readonly ps_process_select_options="-C java -C jsvc" +fi + +# output field: pid, thread id(lwp), pcpu, user +# order by pcpu(percentage of cpu usage) +findBusyJavaThreadsByPs() { + # 1. sort by %cpu by ps option `--sort -pcpu` + # 2. use wide output(unlimited width) by ps option `-ww` + # avoid trunk user column to username_fo+ or $uid alike + local -a ps_cmd_line=(ps $ps_process_select_options -wwLo pid,lwp,pcpu,user --sort -pcpu --no-headers) + local -r ps_out="$("${ps_cmd_line[@]}")" + if [ -n "$store_dir" ]; then + echo "$ps_out" | logAndCat "${ps_cmd_line[@]}" > "${store_file_prefix}$(( i + 1 ))_ps" + fi + + echo "$ps_out" | head -n "${count}" +} + +# top with output field: thread id, %cpu +__top_threadId_cpu() { + # 1. sort by %cpu by top option `-o %CPU` + # unfortunately, top version 3.2 does not support -o option(supports from top version 3.3+), + # use + # HOME="$tmp_store_dir" top -H -b -n 1 + # combined + # sort + # instead of + # HOME="$tmp_store_dir" top -H -b -n 1 -o '%CPU' + # 2. change HOME env var when run top, + # so as to prevent top command output format being change by .toprc user config file unexpectedly + # 3. use option `-d 0.5`(update interval 0.5 second) and `-n 2`(update 2 times), + # and use second time update data to get cpu percentage of thread in 0.5 second interval + # 4. top v3.3, there is 1 black line between 2 update; + # but top v3.2, there is 2 blank lines between 2 update! + local -a top_cmd_line=(top -H -b -d $top_delay -n 2) + local -r top_out=$(HOME="$tmp_store_dir" "${top_cmd_line[@]}") + if [ -n "$store_dir" ]; then + echo "$top_out" | logAndCat "${top_cmd_line[@]}" > "${store_file_prefix}$(( i + 1 ))_top" + fi + + echo "$top_out" | + awk 'BEGIN { blockIndex = 0; currentLineHasText = 0; prevLineHasText = 0; } { + currentLineHasText = ($0 != "") + if (prevLineHasText && !currentLineHasText) + blockIndex++ # from text line to empty line, increase block index + + if (blockIndex == 3 && ($NF == "java" || $NF == "jsvc")) # $NF(last field) is command field + # only print 4th text block(blockIndex == 3), aka. process info of second top update + print $1 " " $9 # $1 is thread id field, $9 is %cpu field + + prevLineHasText = currentLineHasText # update prevLineHasText + }' | sort -k2,2nr +} + +__complete_pid_user_by_ps() { + # ps output field: pid, thread id(lwp), user + local -a ps_cmd_line=(ps $ps_process_select_options -wwLo pid,lwp,user --no-headers) + local -r ps_out="$("${ps_cmd_line[@]}")" + if [ -n "$store_dir" ]; then + echo "$ps_out" | logAndCat "${ps_cmd_line[@]}" > "${store_file_prefix}$(( i + 1 ))_ps" + fi + + local idx=0 + local -a line + while IFS=" " read -a line ; do + (( idx < count )) || break + + local threadId="${line[0]}" + local pcpu="${line[1]}" + + # output field: pid, threadId, pcpu, user + local output_fields="$( echo "$ps_out" | + awk -v "threadId=$threadId" -v "pcpu=$pcpu" '$2==threadId { + printf "%s %s %s %s\n", $1, threadId, pcpu, $3; exit + }' )" + if [ -n "$output_fields" ]; then + (( idx++ )) + echo "$output_fields" + fi + done +} + +# output format is same as function findBusyJavaThreadsByPs +findBusyJavaThreadsByTop() { + __top_threadId_cpu | __complete_pid_user_by_ps +} + +printStackOfThreads() { + local -a line + local idx=0 + while IFS=" " read -a line ; do + local pid="${line[0]}" + local threadId="${line[1]}" + local threadId0x="0x`printf %x ${threadId}`" + local pcpu="${line[2]}" + local user="${line[3]}" + + (( idx++ )) + local jstackFile="${store_file_prefix}$(( i + 1 ))_jstack_${pid}" + [ -f "${jstackFile}" ] || { + local -a jstack_cmd_line=( "$jstack_path" ${force} $mix_native_frames $more_lock_info ${pid} ) + if [ "${user}" == "${USER}" ]; then + # run without sudo, when java process user is current user + logAndRun "${jstack_cmd_line[@]}" > ${jstackFile} + elif [ $UID == 0 ]; then + # if java process user is not current user, must run jstack with sudo + logAndRun sudo -u "${user}" "${jstack_cmd_line[@]}" > ${jstackFile} + else + # current user is not root user, so can not run with sudo; print error message and rerun suggestion + redPrint "[$idx] Fail to jstack busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user})." + redPrint "User of java process($user) is not current user($USER), need sudo to rerun:" + yellowPrint " sudo ${COMMAND_LINE[@]}" + normalPrint + continue + fi || { + redPrint "[$idx] Fail to jstack busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user})." + normalPrint + rm "${jstackFile}" &> /dev/null + continue + } + } + + bluePrint "[$idx] Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user}):" + + if [ -n "$mix_native_frames" ]; then + local sed_script="/--------------- $threadId ---------------/,/^---------------/ { + /--------------- $threadId ---------------/b # skip first separator line + /^---------------/d # delete second separator line + p + }" + elif [ -n "$force" ]; then + local sed_script="/^Thread ${threadId}:/,/^$/ { + /^$/d; p # delete end separator line + }" + else + local sed_script="/ nid=${threadId0x} /,/^$/ { + /^$/d; p # delete end separator line + }" + fi + { + sed "$sed_script" -n ${jstackFile} + echo + } | tee ${append_file:+-a "$append_file"} ${store_dir:+-a "${store_file_prefix}$PROG"} + done +} + +################################################################################ +# Main +################################################################################ + +main() { + local i + # if update_count <= 0, infinite loop till user interrupted (eg: CTRL+C) + for (( i = 0; update_count <= 0 || i < update_count; ++i )); do + (( i > 0 )) && sleep "$update_delay" + + [ -n "$append_file" -o -n "$store_dir" ] && headInfo | tee ${append_file:+-a "$append_file"} ${store_dir:+-a "${store_file_prefix}$PROG"} > /dev/null + (( update_count != 1 )) && headInfo + + if $use_ps; then + findBusyJavaThreadsByPs + else + findBusyJavaThreadsByTop + fi | printStackOfThreads + done +} + +main diff --git a/show-busy-java-threads.sh b/show-busy-java-threads.sh deleted file mode 100755 index 782d7c27..00000000 --- a/show-busy-java-threads.sh +++ /dev/null @@ -1,157 +0,0 @@ -#!/bin/bash -# @Function -# Find out the highest cpu consumed threads of java, and print the stack of these threads. -# -# @Usage -# $ ./show-busy-java-threads.sh -# -# @author Jerry Lee - -readonly PROG=`basename $0` -readonly -a COMMAND_LINE=("$0" "$@") - -usage() { - local out - [ -n "$1" -a "$1" != 0 ] && out=/dev/stderr || out=/dev/stdout - - > $out cat < /dev/null; then - [ -z "$JAVA_HOME" ] && { - redEcho "Error: jstack not found on PATH!" 1>&2 - exit 1 - } - ! [ -f "$JAVA_HOME/bin/jstack" ] && { - redEcho "Error: jstack not found on PATH and \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) file does NOT exists!" 1>&2 - exit 1 - } - ! [ -x "$JAVA_HOME/bin/jstack" ] && { - redEcho "Error: jstack not found on PATH and \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) is NOT executalbe!" 1>&2 - exit 1 - } - export PATH="$JAVA_HOME/bin:$PATH" -fi - -readonly uuid=`date +%s`_${RANDOM}_$$ - -cleanupWhenExit() { - rm /tmp/${uuid}_* &> /dev/null -} -trap "cleanupWhenExit" EXIT - -printStackOfThreads() { - local line - local count=1 - while IFS=" " read -a line ; do - local pid=${line[0]} - local threadId=${line[1]} - local threadId0x="0x`printf %x ${threadId}`" - local user=${line[2]} - local pcpu=${line[4]} - - local jstackFile=/tmp/${uuid}_${pid} - - [ ! -f "${jstackFile}" ] && { - { - if [ "${user}" == "${USER}" ]; then - jstack ${force} ${pid} > ${jstackFile} - else - if [ $UID == 0 ]; then - sudo -u ${user} jstack ${force} ${pid} > ${jstackFile} - else - redEcho "[$((count++))] Fail to jstack Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user})." - redEcho "User of java process($user) is not current user($USER), need sudo to run again:" - yellowEcho " sudo ${COMMAND_LINE[@]}" - echo - continue - fi - fi - } || { - redEcho "[$((count++))] Fail to jstack Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user})." - echo - rm ${jstackFile} - continue - } - } - blueEcho "[$((count++))] Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user}):" - sed "/nid=${threadId0x} /,/^$/p" -n ${jstackFile} - done -} - - -ps -Leo pid,lwp,user,comm,pcpu --no-headers | { - [ -z "${pid}" ] && - awk '$4=="java"{print $0}' || - awk -v "pid=${pid}" '$1==pid,$4=="java"{print $0}' -} | sort -k5 -r -n | head --lines "${count}" | printStackOfThreads diff --git a/show-busy-java-threads.sh b/show-busy-java-threads.sh new file mode 120000 index 00000000..2c3d2d79 --- /dev/null +++ b/show-busy-java-threads.sh @@ -0,0 +1 @@ +show-busy-java-threads \ No newline at end of file diff --git a/show-duplicate-java-classes b/show-duplicate-java-classes index 1f06cb0d..66c868d2 100755 --- a/show-duplicate-java-classes +++ b/show-duplicate-java-classes @@ -1,5 +1,16 @@ #!/usr/bin/env python # -*- coding: utf-8 -*- +# @Function +# Find duplicate class among java libs. +# +# @Usage +# $ show-duplicate-java-classes # find jars from current dir +# $ show-duplicate-java-classes path/to/lib_dir1 /path/to/lib_dir2 +# $ show-duplicate-java-classes -c path/to/class_dir1 -c /path/to/class_dir2 +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#-show-duplicate-java-classes +# @author tg123 (farmer1992 at gmail dot com) +# @author Jerry Lee (oldratlee at gmail dot com) __author__ = 'tg123' @@ -61,7 +72,7 @@ def print_class_paths(class_paths): print("class paths to find:") print("=" * 80) for idx, class_path in enumerate(class_paths): - print(("%-3d: %s" % (idx + 1, class_path))) + print("%-3d: %s" % (idx + 1, class_path)) if __name__ == '__main__': diff --git a/svn-merge-stop-on-copy.sh b/svn-merge-stop-on-copy.sh index d0e05891..4a50bf90 100755 --- a/svn-merge-stop-on-copy.sh +++ b/svn-merge-stop-on-copy.sh @@ -7,8 +7,9 @@ # $ ./svnmerge.sh [target branch] # if no target branch, merge to current svn direcotry # +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/vcs.md#-svn-merge-stop-on-copysh # @author jiangjizhong(@jzwlqx) -# @author Jerry Lee +# @author Jerry Lee (oldratlee at gmail dot com) PROG=`basename $0` diff --git a/swtrunk.sh b/swtrunk.sh index 0fa246fd..e16f97ed 100755 --- a/swtrunk.sh +++ b/swtrunk.sh @@ -5,19 +5,18 @@ # @Usage # $ ./swtrunk.sh [...] # -# @author Jerry Lee +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/vcs.md#-swtrunksh +# @author Jerry Lee (oldratlee at gmail dot com) + +# NOTE: $'foo' is the escape sequence syntax of bash +readonly ec=$'\033' # escape char +readonly eend=$'\033[0m' # escape end colorEcho() { local color=$1 shift - if [ -c /dev/stdout ] ; then - # if stdout is console, turn on color output. - echo -ne "\033[1;${color}m" - echo -n "$@" - echo -e "\033[0m" - else - echo "$@" - fi + # if stdout is console, turn on color output. + [ -t 1 ] && echo "$ec[1;${color}m$@$eend" || echo "$@" } redEcho() { diff --git a/tcp-connection-state-counter b/tcp-connection-state-counter new file mode 100755 index 00000000..47855d20 --- /dev/null +++ b/tcp-connection-state-counter @@ -0,0 +1,23 @@ +#!/bin/bash +# @Function +# show count of tcp connection stat. +# +# @Usage +# $ ./tcp-connection-state-counter +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-tcp-connection-state-counter +# @author Jerry Lee (oldratlee at gmail dot com) +# @author @sunuslee (sunuslee at gmail dot com) + +# On MacOS, netstat need to using -p tcp to get only tcp output. +uname | grep Darwin -q && option_for_mac="-ptcp" + +netstat -tna $option_for_mac | awk 'NR > 2 { + s[$NF]++ +} + +END { + for(v in s) { + printf "%-11s %s\n", v, s[v] + } +}' | sort -nr -k2,2 diff --git a/tcp-connection-state-counter.sh b/tcp-connection-state-counter.sh deleted file mode 100755 index 79b7ea6e..00000000 --- a/tcp-connection-state-counter.sh +++ /dev/null @@ -1,22 +0,0 @@ -#!/bin/bash -# @Function -# show count of tcp connection stat. -# -# @Usage -# $ ./tcp-connection-state-counter.sh -# -# @author Jerry Lee(@oldratlee) -# @author @sunuslee - -# On MacOS, netstat need to using -p tcp to get only tcp output. -uname | grep Darwin -q && option_for_mac="-ptcp" - -netstat -tna $option_for_mac | awk 'NR > 2 { - s[$NF]++ -} - -END { - for(v in s) { - printf "%-11s %s\n", v, s[v] - } -}' | sort -nr -k2,2 diff --git a/tcp-connection-state-counter.sh b/tcp-connection-state-counter.sh new file mode 120000 index 00000000..60b5928f --- /dev/null +++ b/tcp-connection-state-counter.sh @@ -0,0 +1 @@ +tcp-connection-state-counter \ No newline at end of file diff --git a/test-cases/parseOpts-test.sh b/test-cases/parseOpts-test.sh index 81d300f8..95271746 100755 --- a/test-cases/parseOpts-test.sh +++ b/test-cases/parseOpts-test.sh @@ -6,20 +6,18 @@ BASE=`dirname $0` ################################################# -# Utils Methods +# Util Functions ################################################# +# NOTE: $'foo' is the escape sequence syntax of bash +readonly ec=$'\033' # escape char +readonly eend=$'\033[0m' # escape end + colorEcho() { local color=$1 shift - if [ -c /dev/stdout ] ; then - # if stdout is console, turn on color output. - echo -ne "\033[1;${color}m" - echo -n "$@" - echo -e "\033[0m" - else - echo "$@" - fi + # if stdout is console, turn on color output. + [ -t 1 ] && echo "$ec[1;${color}m$@$eend" || echo "$@" } redEcho() { diff --git a/test-cases/shunit2-lib b/test-cases/shunit2-lib new file mode 160000 index 00000000..ebc4baa0 --- /dev/null +++ b/test-cases/shunit2-lib @@ -0,0 +1 @@ +Subproject commit ebc4baa08f045b7ef0f45c4b7d6f34f08d732f3d diff --git a/xpf b/xpf index 47a8b2d5..a51e9831 100755 --- a/xpf +++ b/xpf @@ -1,11 +1,13 @@ #!/bin/bash # @Function -# Open file in file explorer. -# same as xpl --selected [FILE] +# Open file in file explorer, file is selected. +# same as xpl --selected [file [file ...] ] # # @Usage # $ ./xpf file # -# @author Jerry Lee -BASE=`dirname $0` -. $BASE/xpl "$@" +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-xpl-and-xpf +# @author Jerry Lee (oldratlee at gmail dot com) + +BASE="$(dirname "$0")" +source "$BASE/xpl" "$@" diff --git a/xpl b/xpl index 44742fdb..adafa351 100755 --- a/xpl +++ b/xpl @@ -3,13 +3,18 @@ # Open file in file explorer. # # @Usage -# $ ./xpf file +# $ ./xpf [file [file ...] ] # -# @author Jerry Lee +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#-xpl-and-xpf +# @author Jerry Lee (oldratlee at gmail dot com) PROG=`basename $0` usage() { + [ -n "$1" -a "$1" != 0 ] && local out=/dev/stderr || local out=/dev/stdout + + [ $# -gt 1 ] && { echo "$2"; echo; } > $out + cat <