diff --git a/README.md b/README.md index 7e8251b8..ead32cb0 100644 --- a/README.md +++ b/README.md @@ -32,11 +32,11 @@ source <(curl -fsSL https://raw.githubusercontent.com/oldratlee/useful-scripts/m ### :coffee: [`Java`相关脚本](docs/java.md) -1. [show-busy-java-threads.sh](docs/java.md#beer-show-busy-java-threadssh) +1. [show-busy-java-threads](docs/java.md#beer-show-busy-java-threads) 用于快速排查`Java`的`CPU`性能问题(`top us`值过高)，自动查出运行的`Java`进程中消耗`CPU`多的线程，并打印出其线程栈，从而确定导致性能问题的方法调用。 1. [show-duplicate-java-classes](docs/java.md#beer-show-duplicate-java-classes) 找出`jar`文件和`class`目录中的重复类。用于排查`Java`类冲突问题。 -1. [find-in-jars.sh](docs/java.md#beer-find-in-jarssh) +1. [find-in-jars](docs/java.md#beer-find-in-jars) 在目录下所有`jar`文件里，查找类或资源文件。 ### :shell: [`Shell`相关脚本](docs/shell.md) @@ -51,14 +51,14 @@ source <(curl -fsSL https://raw.githubusercontent.com/oldratlee/useful-scripts/m 按行彩色输出参数，方便人眼查看。 1. [ap and rp](docs/shell.md#beer-ap-and-rp) 批量转换文件路径为绝对路径/相对路径，会自动跟踪链接并规范化路径。 -1. [tcp-connection-state-counter.sh](docs/shell.md#beer-tcp-connection-state-countersh) +1. [tcp-connection-state-counter](docs/shell.md#beer-tcp-connection-state-counter) 统计各个`TCP`连接状态的个数。用于方便排查系统连接负荷问题。 1. [xpl and xpf](docs/shell.md#beer-xpl-and-xpf) 在命令行中快速完成 在文件浏览器中 打开/选中 指定的文件或文件夹的操作，优化命令行与其它应用之间的操作流。 `Shell`开发/测试加强： -1. [echo-args.sh](docs/shell.md#beer-echo-argssh) +1. [echo-args](docs/shell.md#beer-echo-args) 输出脚本收到的参数，在控制台运行时，把参数值括起的括号显示成 **红色**，方便人眼查看。用于调试脚本参数输入。 1. [console-text-color-themes.sh](docs/shell.md#beer-console-text-color-themessh) 显示`Terminator`的全部文字彩色组合的效果及其打印方式，用于开发`Shell`的彩色输出。 diff --git a/a2l b/a2l index c859bc3f..22edb562 100755 --- a/a2l +++ b/a2l @@ -26,5 +26,5 @@ readonly -a ECHO_COLORS=(31 32 37 34 33 35 36) COUNT=0 for a; do - colorEcho "${ECHO_COLORS[$((COUNT++ % ${#ECHO_COLORS[@]}))]}" "$a" + colorEcho "${ECHO_COLORS[COUNT++ % ${#ECHO_COLORS[@]}]}" "$a" done diff --git a/coat b/coat index 4b18b6d9..162f4d20 100755 --- a/coat +++ b/coat @@ -20,7 +20,7 @@ readonly eend=$'\033[0m' # escape end readonly -a ECHO_COLORS=(31 32 37 34 33 35 36) COUNT=0 colorEcho() { - local color="${ECHO_COLORS[$((COUNT++ % ${#ECHO_COLORS[@]}))]}" + local color="${ECHO_COLORS[COUNT++ % ${#ECHO_COLORS[@]}]}" # check isatty in bash https://stackoverflow.com/questions/10022323 # if stdout is console, turn on color output. [ -t 1 ] && echo "$ec[1;${color}m$@$eend" || echo "$@" diff --git a/docs/install.md b/docs/install.md index 78d404a9..f8450840 100644 --- a/docs/install.md +++ b/docs/install.md @@ -52,19 +52,19 @@ unzip release.zip 下载和运行单个文件 ------------------- -以[`show-busy-java-threads.sh`](https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads.sh)为例。 +以[`show-busy-java-threads`](https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads)为例。 ### `curl`文件直接用`bash`运行 ```bash -curl -sLk 'https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads.sh' | bash +curl -sLk 'https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads' | bash ``` ### 下载单个文件 ```bash -wget --no-check-certificate https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads.sh -chmod +x show-busy-java-threads.sh +wget --no-check-certificate https://raw.github.com/oldratlee/useful-scripts/release/show-busy-java-threads +chmod +x show-busy-java-threads -./show-busy-java-threads.sh +./show-busy-java-threads ``` diff --git a/docs/java.md b/docs/java.md index bd967c29..0bc883a7 100644 --- a/docs/java.md +++ b/docs/java.md @@ -5,7 +5,7 @@ -- [:beer: show-busy-java-threads.sh](#beer-show-busy-java-threadssh) +- [:beer: show-busy-java-threads](#beer-show-busy-java-threads) - [用法](#%E7%94%A8%E6%B3%95) - [示例](#%E7%A4%BA%E4%BE%8B) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85) @@ -17,14 +17,14 @@ - [`Android`开发场景使用说明](#android%E5%BC%80%E5%8F%91%E5%9C%BA%E6%99%AF%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E) - [示例](#%E7%A4%BA%E4%BE%8B-1) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-1) -- [:beer: find-in-jars.sh](#beer-find-in-jarssh) +- [:beer: find-in-jars](#beer-find-in-jars) - [用法](#%E7%94%A8%E6%B3%95-2) - [示例](#%E7%A4%BA%E4%BE%8B-2) - [参考资料](#%E5%8F%82%E8%80%83%E8%B5%84%E6%96%99) -:beer: [show-busy-java-threads.sh](../show-busy-java-threads.sh) +:beer: [show-busy-java-threads](../show-busy-java-threads) ---------------------- 用于快速排查`Java`的`CPU`性能问题(`top us`值过高)，自动查出运行的`Java`进程中消耗`CPU`多的线程，并打印出其线程栈，从而确定导致性能问题的方法调用。 @@ -46,19 +46,19 @@ PS，如何操作可以参见[@bluedavy](http://weibo.com/bluedavy)的《分布 ### 用法 ```bash -show-busy-java-threads.sh +show-busy-java-threads # 从所有运行的Java进程中找出最消耗CPU的线程（缺省5个），打印出其线程栈 # 缺省会自动从所有的Java进程中找出最消耗CPU的线程，这样用更方便 # 当然你可以手动指定要分析的Java进程Id，以保证只会显示出那个你关心的Java进程的信息 -show-busy-java-threads.sh -p <指定的Java进程Id> +show-busy-java-threads -p <指定的Java进程Id> -show-busy-java-threads.sh -c <要显示的线程栈数> +show-busy-java-threads -c <要显示的线程栈数> -show-busy-java-threads.sh <重复执行的间隔秒数> [<重复执行的次数>] +show-busy-java-threads <重复执行的间隔秒数> [<重复执行的次数>] # 多次执行；这2个参数的使用方式类似vmstat命令 -show-busy-java-threads.sh -a <输出记录到的文件> +show-busy-java-threads -a <输出记录到的文件> # 记录到文件以方便回溯查看 ############################## @@ -66,30 +66,30 @@ show-busy-java-threads.sh -a <输出记录到的文件> ############################## # 如果Java进程的用户 与 执行脚本的当前用户 不同，则jstack不了这个Java进程 # 为了能切换到Java进程的用户，需要加sudo来执行，即可以解决： -sudo show-busy-java-threads.sh +sudo show-busy-java-threads -show-busy-java-threads.sh -s <指定jstack命令的全路径> +show-busy-java-threads -s <指定jstack命令的全路径> # 对于sudo方式的运行，JAVA_HOME环境变量不能传递给root， # 而root用户往往没有配置JAVA_HOME且不方便配置， # 显式指定jstack命令的路径就反而显得更方便了 # -m选项：执行jstack命令时加上-m选项，显示上Native的栈帧，一般应用排查不需要使用 -show-busy-java-threads.sh -m +show-busy-java-threads -m # -F选项：执行jstack命令时加上 -F 选项（如果直接jstack无响应时，用于强制jstack），一般情况不需要使用 -show-busy-java-threads.sh -F +show-busy-java-threads -F # -l选项：执行jstack命令时加上 -l 选项，显示上更多相关锁的信息，一般情况不需要使用 # 注意：和 -m -F 选项一起使用时，可能会大大增加jstack操作的耗时 -show-busy-java-threads.sh -l +show-busy-java-threads -l # 帮助信息 -$ show-busy-java-threads.sh -h -Usage: show-busy-java-threads.sh [OPTION]... [delay [count]] +$ show-busy-java-threads -h +Usage: show-busy-java-threads [OPTION]... [delay [count]] Find out the highest cpu consumed threads of java, and print the stack of these threads. Example: - show-busy-java-threads.sh # show busy java threads info - show-busy-java-threads.sh 1 # update every 1 seconds, (stop by eg: CTRL+C) - show-busy-java-threads.sh 3 10 # update every 3 seconds, update 10 times + show-busy-java-threads # show busy java threads info + show-busy-java-threads 1 # update every 1 seconds, (stop by eg: CTRL+C) + show-busy-java-threads 3 10 # update every 3 seconds, update 10 times Options: -p, --pid find out the highest cpu consumed threads from the specifed java process, @@ -110,7 +110,7 @@ Options: ### 示例 ```bash -$ show-busy-java-threads.sh +$ show-busy-java-threads [1] Busy(57.0%) thread(23355/0x5b3b) stack of java process(23269) under user(admin): "pool-1-thread-1" prio=10 tid=0x000000005b5c5000 nid=0x5b3b runnable [0x000000004062c000] java.lang.Thread.State: RUNNABLE @@ -146,7 +146,7 @@ $ show-busy-java-threads.sh - `AppMonitorDataAvgScheduler.run`调用`DateFormat.format`次数比较频繁。 - `DateFormat.format`比较慢。（这个可以由`DateFormat.format`的实现确定。） -多执行几次`show-busy-java-threads.sh`，如果上面情况高概率出现，则可以确定上面的判定。 +多执行几次`show-busy-java-threads`，如果上面情况高概率出现，则可以确定上面的判定。 因为调用越少代码执行越快，则出现在线程栈的概率就越低。 脚本有自动多次执行的功能，指定 重复执行的间隔秒数/重复执行的次数 参数。 @@ -321,7 +321,7 @@ class paths to find: [tgic](https://github.com/tg123)提供此脚本。友情贡献者的链接[commandlinefu.cn](http://commandlinefu.cn/)|[微博linux命令行精选](http://weibo.com/u/2674868673) -:beer: [find-in-jars.sh](../find-in-jars.sh) +:beer: [find-in-jars](../find-in-jars) ---------------------- 在当前目录下所有`jar`文件里，查找类或资源文件。 @@ -331,27 +331,27 @@ class paths to find: ```bash # 在当前目录下所有`jar`文件里，查找类或资源文件。 -find-in-jars.sh 'log4j\.properties' -find-in-jars.sh 'log4j\.xml$' -find-in-jars.sh log4j\\.xml$ # 和上面命令一样，Shell转义的不同写法而已 -find-in-jars.sh 'log4j(\.properties|\.xml)$' +find-in-jars 'log4j\.properties' +find-in-jars 'log4j\.xml$' +find-in-jars log4j\\.xml$ # 和上面命令一样，Shell转义的不同写法而已 +find-in-jars 'log4j(\.properties|\.xml)$' # -d选项 指定 查找目录（覆盖缺省的当前目录） -find-in-jars.sh 'log4j\.properties$' -d /path/to/find/directory +find-in-jars 'log4j\.properties$' -d /path/to/find/directory # 支持多个查找目录 -find-in-jars.sh 'log4j\.properties' -d /path/to/find/directory1 -d /path/to/find/directory2 +find-in-jars 'log4j\.properties' -d /path/to/find/directory1 -d /path/to/find/directory2 # 帮助信息 -$ find-in-jars.sh -h -Usage: find-in-jars.sh [OPTION]... PATTERN +$ find-in-jars -h +Usage: find-in-jars [OPTION]... PATTERN Find file in the jar files under specified directory(recursive, include subdirectory). The pattern default is *extended* regex. Example: - find-in-jars.sh 'log4j\.properties' - find-in-jars.sh '^log4j(\.properties|\.xml)$' # search file log4j.properties/log4j.xml at zip root - find-in-jars.sh 'log4j\.properties$' -d /path/to/find/directory - find-in-jars.sh 'log4j\.properties' -d /path/to/find/dir1 -d /path/to/find/dir2 + find-in-jars 'log4j\.properties' + find-in-jars '^log4j(\.properties|\.xml)$' # search file log4j.properties/log4j.xml at zip root + find-in-jars 'log4j\.properties$' -d /path/to/find/directory + find-in-jars 'log4j\.properties' -d /path/to/find/dir1 -d /path/to/find/dir2 Options: -d, --dir the directory that find jar files, default is current directory. @@ -369,7 +369,7 @@ Options: ```bash # 在当前目录下的所有Jar文件中，查找出 log4j.properties文件 -$ find-in-jars.sh 'log4j\.properties$' +$ find-in-jars 'log4j\.properties$' ./hadoop-core-0.20.2-cdh3u3.jar!log4j.properties # 查找出 以Service结尾的类 @@ -379,7 +379,7 @@ $ ./find-in-jars 'Service.class$' ...... # 在指定的多个目录的Jar文件中，查找出 properties文件 -$ find-in-jars.sh '\.properties$' -d ../WEB-INF/lib -d ../deploy/lib | grep -v '/pom\.properties$' +$ find-in-jars '\.properties$' -d ../WEB-INF/lib -d ../deploy/lib | grep -v '/pom\.properties$' ../WEB-INF/lib/aspectjtools-1.6.2.jar!org/aspectj/ajdt/ajc/messages.properties ../WEB-INF/lib/aspectjtools-1.6.2.jar!org/aspectj/ajdt/internal/compiler/parser/readableNames.properties ../WEB-INF/lib/aspectjweaver-1.8.8.jar!org/aspectj/weaver/XlintDefault.properties diff --git a/docs/shell.md b/docs/shell.md index 5d60bf97..ab1ecb9c 100644 --- a/docs/shell.md +++ b/docs/shell.md @@ -15,7 +15,7 @@ - [示例](#%E7%A4%BA%E4%BE%8B-2) - [:beer: ap and rp](#beer-ap-and-rp) - [示例](#%E7%A4%BA%E4%BE%8B-3) - - [:beer: tcp-connection-state-counter.sh](#beer-tcp-connection-state-countersh) + - [:beer: tcp-connection-state-counter](#beer-tcp-connection-state-counter) - [用法](#%E7%94%A8%E6%B3%95) - [示例](#%E7%A4%BA%E4%BE%8B-4) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85) @@ -24,7 +24,7 @@ - [示例](#%E7%A4%BA%E4%BE%8B-5) - [贡献者](#%E8%B4%A1%E7%8C%AE%E8%80%85-1) - [`Shell`开发/测试加强](#shell%E5%BC%80%E5%8F%91%E6%B5%8B%E8%AF%95%E5%8A%A0%E5%BC%BA) - - [:beer: echo-args.sh](#beer-echo-argssh) + - [:beer: echo-args](#beer-echo-args) - [示例](#%E7%A4%BA%E4%BE%8B-6) - [使用方式](#%E4%BD%BF%E7%94%A8%E6%96%B9%E5%BC%8F) - [:beer: console-text-color-themes.sh](#beer-console-text-color-themessh) @@ -162,8 +162,8 @@ B.java # zsh支持 **/* 跨目录glob，可以方便搜索，但是输出内容是空格分隔的不方便查看。 # 把参数按行输出方便查看 或是 grep $ a2l **/*.sh +console-text-color-themes.sh swtrunk.sh -tcp-connection-state-counter.sh test-cases/parseOpts-test.sh test-cases/self-installer.sh ... @@ -203,7 +203,7 @@ $ rp /home /etc/../etc /home/admin ../../etc ``` -:beer: [tcp-connection-state-counter.sh](../tcp-connection-state-counter.sh) +:beer: [tcp-connection-state-counter](../tcp-connection-state-counter) ---------------------- 统计各个`TCP`连接状态的个数。 @@ -218,13 +218,13 @@ $ rp /home /etc/../etc /home/admin ### 用法 ```bash -tcp-connection-state-counter.sh +tcp-connection-state-counter ``` ### 示例 ```bash -$ tcp-connection-state-counter.sh +$ tcp-connection-state-counter ESTABLISHED 290 TIME_WAIT 212 SYN_SENT 17 @@ -276,7 +276,7 @@ xpf /path/to/dir1 /path/to/foo1.txt `Shell`开发/测试加强 ==================================== -:beer: [echo-args.sh](../echo-args.sh) +:beer: [echo-args](../echo-args) ---------------------- 在编写脚本时，常常要确认输入参数是否是期望的：参数个数，参数值（可能包含有人眼不容易发现的空格问题）。 @@ -287,8 +287,8 @@ xpf /path/to/dir1 /path/to/foo1.txt ### 示例 ```bash -$ ./echo-args.sh 1 " 2 foo " "3 3" -0/3: [./echo-args.sh] +$ ./echo-args 1 " 2 foo " "3 3" +0/3: [./echo-args] 1/3: [1] 2/3: [ 2 foo ] 3/3: [3 3] @@ -299,7 +299,7 @@ $ ./echo-args.sh 1 " 2 foo " "3 3" 需要查看某个脚本（实际上也可以是其它的可执行程序）输出参数时，可以这么做： * 把要查看脚本重命名。 -* 建一个`echo-args.sh`脚本的符号链接到要查看参数的脚本的位置，名字和查看脚本一样。 +* 建一个`echo-args`脚本的符号链接到要查看参数的脚本的位置，名字和查看脚本一样。 这样可以不改其它的程序，查看到输入参数的信息。 diff --git a/echo-args b/echo-args new file mode 100755 index 00000000..b02fcab2 --- /dev/null +++ b/echo-args @@ -0,0 +1,27 @@ +#!/bin/bash +# @Function +# print arguments in human and debugging friendly style. +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#beer-echo-args +# @author Jerry Lee (oldratlee at gmail dot com) + +readonly ebegin=$'\033[1;31m' # escape begin, red color +readonly eend=$'\033[0m' # escape end + +echoArg() { + local index=$1 + local count=$2 + local value=$3 + + # if stdout is console, turn on color output. + [ -t 1 ] && + echo "$index/$count: $ebegin[$eend$value$ebegin]$eend" || + echo "$index/$count: [$value]" +} + + +echoArg 0 $# "$0" +idx=1 +for a ; do + echoArg $((idx++)) $# "$a" +done diff --git a/echo-args.sh b/echo-args.sh deleted file mode 100755 index 1010de0d..00000000 --- a/echo-args.sh +++ /dev/null @@ -1,27 +0,0 @@ -#!/bin/bash -# @Function -# print arguments in human and debug friendly style. -# -# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#beer-echo-argssh -# @author Jerry Lee (oldratlee at gmail dot com) - -readonly ebegin=$'\033[1;31m' # escape begin, red color -readonly eend=$'\033[0m' # escape end - -echoArg() { - local index=$1 - local count=$2 - local value=$3 - - # if stdout is console, turn on color output. - [ -t 1 ] && - echo "$index/$count: $ebegin[$eend$value$ebegin]$eend" || - echo "$index/$count: [$value]" -} - - -echoArg 0 $# "$0" -idx=1 -for a ; do - echoArg $((idx++)) $# "$a" -done diff --git a/echo-args.sh b/echo-args.sh new file mode 120000 index 00000000..32c33811 --- /dev/null +++ b/echo-args.sh @@ -0,0 +1 @@ +echo-args \ No newline at end of file diff --git a/find-in-jars b/find-in-jars new file mode 100755 index 00000000..11b64994 --- /dev/null +++ b/find-in-jars @@ -0,0 +1,174 @@ +#!/bin/bash +# @Function +# Find file in the jar files under current directory +# +# @Usage +# $ find-in-jars 'log4j\.properties' +# $ find-in-jars '^log4j(\.properties|\.xml)$' +# $ find-in-jars 'log4j\.properties$' -d /path/to/find/directory +# $ find-in-jars 'log4j\.properties' -d /path/to/find/directory1 -d /path/to/find/directory2 +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#beer-find-in-jars +# @author Jerry Lee (oldratlee at gmail dot com) + +readonly PROG=`basename $0` + +usage() { + [ -n "$1" -a "$1" != 0 ] && local out=/dev/stderr || local out=/dev/stdout + + > $out cat <&2 ; usage 1; } +[ "${#args[@]}" -gt 1 ] && { echo "More than 1 file pattern!" 1>&2 ; usage 1; } +readonly pattern="${args[0]}" + +################################################################################ +# Check the existence of command for listing zip entry! +################################################################################ + +# `zipinfo -1`/`unzip -Z1` is ~25 times faster than `jar tf`, find zipinfo/unzip command first. +# +# How to list files in a zip without extra information in command line +# https://unix.stackexchange.com/a/128304/136953 +if which zipinfo &> /dev/null; then + readonly command_for_list_zip='zipinfo -1' +elif which unzip &> /dev/null; then + readonly command_for_list_zip='unzip -Z1' +else + if ! which jar &> /dev/null; then + [ -z "$JAVA_HOME" ] && { + echo "Error: jar not found on PATH!" 1>&2 + exit 1 + } + ! [ -f "$JAVA_HOME/bin/jar" ] && { + echo "Error: jar not found on PATH and \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) file does NOT exists!" 1>&2 + exit 1 + } + ! [ -x "$JAVA_HOME/bin/jar" ] && { + echo "Error: jar not found on PATH and \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) is NOT executalbe!" 1>&2 + exit 1 + } + export PATH="$JAVA_HOME/bin:$PATH" + fi + + readonly command_for_list_zip='jar tf' +fi + +################################################################################ +# find logic +################################################################################ + +[ -t 1 ] && readonly is_console=true || readonly is_console=false + +# Getting console width using a bash script +# https://unix.stackexchange.com/questions/299067 +$is_console && readonly columns=$(stty size | awk '{print $2}') + +readonly ec=$'\033' # escape char +readonly cr=$'\r' # carriage return + +printResponsiveMessage() { + $is_console || return + + local message="$*" + # http://www.linuxforums.org/forum/red-hat-fedora-linux/142825-how-truncate-string-bash-script.html + echo -n "${message:0:columns}" +} + +clearResponsiveMessage() { + $is_console || return + + # How to delete line with echo? + # https://unix.stackexchange.com/questions/26576 + # + # terminal escapes: http://ascii-table.com/ansi-escape-sequences.php + # In particular, to clear from the cursor position to the beginning of the line: + # echo -e "\033[1K" + # Or everything on the line, regardless of cursor position: + # echo -e "\033[2K" + echo -n "$ec[2K$cr" +} + + +readonly jar_files="$(find "${dirs[@]}" -iname '*.jar')" +readonly total_count="$(echo "$jar_files" | wc -l)" +[ -z "$jar_files" ] && { + echo "No jar found!" 1>&2 + exit 1 +} + +counter=1 +while read jar_file; do + printResponsiveMessage "finding in jar($((counter++))/$total_count): $jar_file" + + $command_for_list_zip "${jar_file}" | grep $regex_mode "$pattern" | while read file; do + clearResponsiveMessage + + echo "${jar_file}"\!"${file}" + done + + clearResponsiveMessage + +done < <(echo "$jar_files") diff --git a/find-in-jars.sh b/find-in-jars.sh deleted file mode 100755 index ef79e46b..00000000 --- a/find-in-jars.sh +++ /dev/null @@ -1,174 +0,0 @@ -#!/bin/bash -# @Function -# Find file in the jar files under current directory -# -# @Usage -# $ find-in-jars.sh 'log4j\.properties' -# $ find-in-jars.sh '^log4j(\.properties|\.xml)$' -# $ find-in-jars.sh 'log4j\.properties$' -d /path/to/find/directory -# $ find-in-jars.sh 'log4j\.properties' -d /path/to/find/directory1 -d /path/to/find/directory2 -# -# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#beer-find-in-jarssh -# @author Jerry Lee (oldratlee at gmail dot com) - -readonly PROG=`basename $0` - -usage() { - [ -n "$1" -a "$1" != 0 ] && local out=/dev/stderr || local out=/dev/stdout - - > $out cat <&2 ; usage 1; } -[ "${#args[@]}" -gt 1 ] && { echo "More than 1 file pattern!" 1>&2 ; usage 1; } -readonly pattern="${args[0]}" - -################################################################################ -# Check the existence of command for listing zip entry! -################################################################################ - -# `zipinfo -1`/`unzip -Z1` is ~25 times faster than `jar tf`, find zipinfo/unzip command first. -# -# How to list files in a zip without extra information in command line -# https://unix.stackexchange.com/a/128304/136953 -if which zipinfo &> /dev/null; then - readonly command_for_list_zip='zipinfo -1' -elif which unzip &> /dev/null; then - readonly command_for_list_zip='unzip -Z1' -else - if ! which jar &> /dev/null; then - [ -z "$JAVA_HOME" ] && { - echo "Error: jar not found on PATH!" 1>&2 - exit 1 - } - ! [ -f "$JAVA_HOME/bin/jar" ] && { - echo "Error: jar not found on PATH and \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) file does NOT exists!" 1>&2 - exit 1 - } - ! [ -x "$JAVA_HOME/bin/jar" ] && { - echo "Error: jar not found on PATH and \$JAVA_HOME/bin/jar($JAVA_HOME/bin/jar) is NOT executalbe!" 1>&2 - exit 1 - } - export PATH="$JAVA_HOME/bin:$PATH" - fi - - readonly command_for_list_zip='jar tf' -fi - -################################################################################ -# find logic -################################################################################ - -[ -t 1 ] && readonly is_console=true || readonly is_console=false - -# Getting console width using a bash script -# https://unix.stackexchange.com/questions/299067 -$is_console && readonly columns=$(stty size | awk '{print $2}') - -readonly ec=$'\033' # escape char -readonly cr=$'\r' # carriage return - -printResponsiveMessage() { - $is_console || return - - local message="$*" - # http://www.linuxforums.org/forum/red-hat-fedora-linux/142825-how-truncate-string-bash-script.html - echo -n "${message:0:columns}" -} - -clearResponsiveMessage() { - $is_console || return - - # How to delete line with echo? - # https://unix.stackexchange.com/questions/26576 - # - # terminal escapes: http://ascii-table.com/ansi-escape-sequences.php - # In particular, to clear from the cursor position to the beginning of the line: - # echo -e "\033[1K" - # Or everything on the line, regardless of cursor position: - # echo -e "\033[2K" - echo -n "$ec[2K$cr" -} - - -readonly jar_files="$(find "${dirs[@]}" -iname '*.jar')" -readonly total_count="$(echo "$jar_files" | wc -l)" -[ -z "$jar_files" ] && { - echo "No jar found!" 1>&2 - exit 1 -} - -counter=1 -while read jar_file; do - printResponsiveMessage "finding in jar($((counter++))/$total_count): $jar_file" - - $command_for_list_zip "${jar_file}" | grep $regex_mode "$pattern" | while read file; do - clearResponsiveMessage - - echo "${jar_file}"\!"${file}" - done - - clearResponsiveMessage - -done < <(echo "$jar_files") diff --git a/find-in-jars.sh b/find-in-jars.sh new file mode 120000 index 00000000..cd6c3a87 --- /dev/null +++ b/find-in-jars.sh @@ -0,0 +1 @@ +find-in-jars \ No newline at end of file diff --git a/rp b/rp index 506bc4df..7018578b 100755 --- a/rp +++ b/rp @@ -24,8 +24,8 @@ argc=$# # Get last argument - relativeTo="${argv[$((argc - 1))]}" - files=( "${argv[@]:0:$((argc - 1))}" ) + relativeTo="${argv[argc - 1]}" + files=( "${argv[@]:0:argc - 1}" ) } [ -f "$relativeTo" ] && relativeTo="$(dirname "$relativeTo")" diff --git a/show-busy-java-threads b/show-busy-java-threads new file mode 100755 index 00000000..cf7d3843 --- /dev/null +++ b/show-busy-java-threads @@ -0,0 +1,245 @@ +#!/bin/bash +# @Function +# Find out the highest cpu consumed threads of java, and print the stack of these threads. +# +# @Usage +# $ ./show-busy-java-threads +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#beer-show-busy-java-threads +# @author Jerry Lee (oldratlee at gmail dot com) +# @author superhj1987 (superhj1987 at 126 dot com) + +readonly PROG="`basename $0`" +readonly -a COMMAND_LINE=("$0" "$@") + +# Check os support! +uname | grep '^Linux' -q || { + echo "$PROG only support Linux, not support `uname` yet!" 1>&2 + exit 2 +} + +# Get corrent current user name via whoami command +# See get https://www.lifewire.com/current-linux-user-whoami-command-3867579 +# Because if use `sudo -u` to run command, env var $USER is not rewrited/correct, just inherited from outside! +readonly USER="`whoami`" + +usage() { + [ -n "$1" -a "$1" != 0 ] && local out=/dev/stderr || local out=/dev/stdout + + > $out cat < find out the highest cpu consumed threads from the specifed java process, + default from all java process. + -c, --count set the thread count to show, default is 5 + -a, --append-file specify the file to append output as log + -s, --jstack-path specify the path of jstack command + -F, --force set jstack to force a thread dump + use when jstack does not respond (process is hung) + -m, --mix-native-frames set jstack to print both java and native frames (mixed mode) + -l, --lock-info set jstack with long listing. Prints additional information about locks + -h, --help display this help and exit + delay the delay between updates in seconds + count the number of updates + delay/count arguments imitates style of vmstat command +EOF + + exit $1 +} + +readonly ARGS=`getopt -n "$PROG" -a -o p:c:a:s:Fmlh -l count:,pid:,append-file:,jstack-path:,force,mix-native-frames,lock-info,help -- "$@"` +[ $? -ne 0 ] && usage 1 +eval set -- "${ARGS}" + +while true; do + case "$1" in + -c|--count) + count="$2" + shift 2 + ;; + -p|--pid) + pid="$2" + shift 2 + ;; + -a|--append-file) + append_file="$2" + shift 2 + ;; + -s|--jstack-path) + jstack_path="$2" + shift 2 + ;; + -F|--force) + force=-F + shift 1 + ;; + -m|--mix-native-frames) + mix_native_frames=-m + shift 1 + ;; + -l|--lock-info) + more_lock_info=-l + shift 1 + ;; + -h|--help) + usage + ;; + --) + shift + break + ;; + esac +done +count=${count:-5} + +update_delay=${1:-0} +[ -z "$1" ] && update_count=1 || update_count=${2:-0} +[ $update_count -lt 0 ] && update_count=0 + +# NOTE: $'foo' is the escape sequence syntax of bash +readonly ec=$'\033' # escape char +readonly eend=$'\033[0m' # escape end + +colorPrint() { + local color=$1 + shift + if [ -t 1 ] ; then + # if stdout is console, turn on color output. + echo "$ec[1;${color}m$@$eend" + else + echo "$@" + fi + + [ -n "$append_file" ] && echo "$@" >> "$append_file" +} + +redPrint() { + colorPrint 31 "$@" +} + +greenPrint() { + colorPrint 32 "$@" +} + +yellowPrint() { + colorPrint 33 "$@" +} + +bluePrint() { + colorPrint 36 "$@" +} + +normalPrint() { + echo "$@" + + [ -n "$append_file" ] && echo "$@" >> "$append_file" +} + +if [ -n "$jstack_path" ]; then + [ ! -x "$jstack_path" ] && { + redPrint "Error: $jstack_path is NOT found/executalbe!" 1>&2 + exit 1 + } +elif which jstack &> /dev/null; then + # Check the existence of jstack command! + jstack_path="`which jstack`" +else + [ -z "$JAVA_HOME" ] && { + redPrint "Error: jstack not found on PATH! Use -s option set jstack path manually." 1>&2 + exit 1 + } + [ ! -f "$JAVA_HOME/bin/jstack" ] && { + redPrint "Error: jstack not found on PATH and \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) file does NOT exists! Use -s option set jstack path manually." 1>&2 + exit 1 + } + [ ! -x "$JAVA_HOME/bin/jstack" ] && { + redPrint "Error: jstack not found on PATH and \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) is NOT executalbe! Use -s option set jstack path manually." 1>&2 + exit 1 + } + jstack_path="$JAVA_HOME/bin/jstack" +fi + +readonly uuid=`date +%s`_${RANDOM}_$$ + +cleanupWhenExit() { + rm /tmp/${uuid}_* &> /dev/null +} +trap "cleanupWhenExit" EXIT + +printStackOfThreads() { + local line + local counter=0 + while IFS=" " read -a line ; do + local pid=${line[0]} + local threadId=${line[1]} + local threadId0x="0x`printf %x ${threadId}`" + local user=${line[2]} + local pcpu=${line[4]} + + ((counter++)) + local jstackFile=/tmp/${uuid}_${pid} + [ ! -f "${jstackFile}" ] && { + { + if [ "${user}" == "${USER}" ]; then + "$jstack_path" ${force} $mix_native_frames $more_lock_info ${pid} > ${jstackFile} + elif [ $UID == 0 ]; then + sudo -u "${user}" "$jstack_path" ${force} $mix_native_frames $more_lock_info ${pid} > ${jstackFile} + else + redPrint "[$counter] Fail to jstack Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user})." + redPrint "User of java process($user) is not current user($USER), need sudo to run again:" + yellowPrint " sudo ${COMMAND_LINE[@]}" + normalPrint + continue + fi + } || { + redPrint "[$counter] Fail to jstack Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user})." + normalPrint + rm ${jstackFile} + continue + } + } + + bluePrint "[$counter] Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user}):" + + if [ -n "$mix_native_frames" ]; then + local sed_script="/--------------- $threadId ---------------/,/^---------------/ { + /--------------- $threadId ---------------/b # skip first seperator line + /^---------------/s/.*// # replace sencond seperator line to empty line + p + }" + elif [ -n "$force" ]; then + local sed_script="/^Thread ${threadId}:/,/^$/p" + else + local sed_script="/nid=${threadId0x} /,/^$/p" + fi + sed "$sed_script" -n ${jstackFile} | tee ${append_file:+-a "$append_file"} + done +} + +headInfo() { + echo ================================================================================ + echo "$(date "+%Y-%m-%d %H:%M:%S.%N") [$((i+1))/$update_count]: ${COMMAND_LINE[@]}" + echo ================================================================================ + echo +} + +# if update_count <= 0, infinite loop till user interupted (eg: CTRL+C) +for ((i = 0; update_count <= 0 || i < update_count; ++i)); do + [ "$i" -gt 0 ] && sleep "$update_delay" + + [ -n "$append_file" ] && headInfo >> "$append_file" + [ "$update_count" -ne 1 ] && headInfo + + ps -Leo pid,lwp,user,comm,pcpu --no-headers | { + [ -z "${pid}" ] && + awk '$4=="java"{print $0}' || + awk -v "pid=${pid}" '$1==pid,$4=="java"{print $0}' + } | sort -k5 -r -n | head -n "${count}" | printStackOfThreads +done diff --git a/show-busy-java-threads.sh b/show-busy-java-threads.sh deleted file mode 100755 index 945429f8..00000000 --- a/show-busy-java-threads.sh +++ /dev/null @@ -1,245 +0,0 @@ -#!/bin/bash -# @Function -# Find out the highest cpu consumed threads of java, and print the stack of these threads. -# -# @Usage -# $ ./show-busy-java-threads.sh -# -# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/java.md#beer-show-busy-java-threadssh -# @author Jerry Lee (oldratlee at gmail dot com) -# @author superhj1987 (superhj1987 at 126 dot com) - -readonly PROG="`basename $0`" -readonly -a COMMAND_LINE=("$0" "$@") - -# Check os support! -uname | grep '^Linux' -q || { - echo "$PROG only support Linux, not support `uname` yet!" 1>&2 - exit 2 -} - -# Get corrent current user name via whoami command -# See get https://www.lifewire.com/current-linux-user-whoami-command-3867579 -# Because if use `sudo -u` to run command, env var $USER is not rewrited/correct, just inherited from outside! -readonly USER="`whoami`" - -usage() { - [ -n "$1" -a "$1" != 0 ] && local out=/dev/stderr || local out=/dev/stdout - - > $out cat < find out the highest cpu consumed threads from the specifed java process, - default from all java process. - -c, --count set the thread count to show, default is 5 - -a, --append-file specify the file to append output as log - -s, --jstack-path specify the path of jstack command - -F, --force set jstack to force a thread dump - use when jstack does not respond (process is hung) - -m, --mix-native-frames set jstack to print both java and native frames (mixed mode) - -l, --lock-info set jstack with long listing. Prints additional information about locks - -h, --help display this help and exit - delay the delay between updates in seconds - count the number of updates - delay/count arguments imitates style of vmstat command -EOF - - exit $1 -} - -readonly ARGS=`getopt -n "$PROG" -a -o p:c:a:s:Fmlh -l count:,pid:,append-file:,jstack-path:,force,mix-native-frames,lock-info,help -- "$@"` -[ $? -ne 0 ] && usage 1 -eval set -- "${ARGS}" - -while true; do - case "$1" in - -c|--count) - count="$2" - shift 2 - ;; - -p|--pid) - pid="$2" - shift 2 - ;; - -a|--append-file) - append_file="$2" - shift 2 - ;; - -s|--jstack-path) - jstack_path="$2" - shift 2 - ;; - -F|--force) - force=-F - shift 1 - ;; - -m|--mix-native-frames) - mix_native_frames=-m - shift 1 - ;; - -l|--lock-info) - more_lock_info=-l - shift 1 - ;; - -h|--help) - usage - ;; - --) - shift - break - ;; - esac -done -count=${count:-5} - -update_delay=${1:-0} -[ -z "$1" ] && update_count=1 || update_count=${2:-0} -[ $update_count -lt 0 ] && update_count=0 - -# NOTE: $'foo' is the escape sequence syntax of bash -readonly ec=$'\033' # escape char -readonly eend=$'\033[0m' # escape end - -colorPrint() { - local color=$1 - shift - if [ -t 1 ] ; then - # if stdout is console, turn on color output. - echo "$ec[1;${color}m$@$eend" - else - echo "$@" - fi - - [ -n "$append_file" ] && echo "$@" >> "$append_file" -} - -redPrint() { - colorPrint 31 "$@" -} - -greenPrint() { - colorPrint 32 "$@" -} - -yellowPrint() { - colorPrint 33 "$@" -} - -bluePrint() { - colorPrint 36 "$@" -} - -normalPrint() { - echo "$@" - - [ -n "$append_file" ] && echo "$@" >> "$append_file" -} - -if [ -n "$jstack_path" ]; then - [ ! -x "$jstack_path" ] && { - redPrint "Error: $jstack_path is NOT found/executalbe!" 1>&2 - exit 1 - } -elif which jstack &> /dev/null; then - # Check the existence of jstack command! - jstack_path="`which jstack`" -else - [ -z "$JAVA_HOME" ] && { - redPrint "Error: jstack not found on PATH! Use -s option set jstack path manually." 1>&2 - exit 1 - } - [ ! -f "$JAVA_HOME/bin/jstack" ] && { - redPrint "Error: jstack not found on PATH and \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) file does NOT exists! Use -s option set jstack path manually." 1>&2 - exit 1 - } - [ ! -x "$JAVA_HOME/bin/jstack" ] && { - redPrint "Error: jstack not found on PATH and \$JAVA_HOME/bin/jstack($JAVA_HOME/bin/jstack) is NOT executalbe! Use -s option set jstack path manually." 1>&2 - exit 1 - } - jstack_path="$JAVA_HOME/bin/jstack" -fi - -readonly uuid=`date +%s`_${RANDOM}_$$ - -cleanupWhenExit() { - rm /tmp/${uuid}_* &> /dev/null -} -trap "cleanupWhenExit" EXIT - -printStackOfThreads() { - local line - local counter=0 - while IFS=" " read -a line ; do - local pid=${line[0]} - local threadId=${line[1]} - local threadId0x="0x`printf %x ${threadId}`" - local user=${line[2]} - local pcpu=${line[4]} - - ((counter++)) - local jstackFile=/tmp/${uuid}_${pid} - [ ! -f "${jstackFile}" ] && { - { - if [ "${user}" == "${USER}" ]; then - "$jstack_path" ${force} $mix_native_frames $more_lock_info ${pid} > ${jstackFile} - elif [ $UID == 0 ]; then - sudo -u "${user}" "$jstack_path" ${force} $mix_native_frames $more_lock_info ${pid} > ${jstackFile} - else - redPrint "[$counter] Fail to jstack Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user})." - redPrint "User of java process($user) is not current user($USER), need sudo to run again:" - yellowPrint " sudo ${COMMAND_LINE[@]}" - normalPrint - continue - fi - } || { - redPrint "[$counter] Fail to jstack Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user})." - normalPrint - rm ${jstackFile} - continue - } - } - - bluePrint "[$counter] Busy(${pcpu}%) thread(${threadId}/${threadId0x}) stack of java process(${pid}) under user(${user}):" - - if [ -n "$mix_native_frames" ]; then - local sed_script="/--------------- $threadId ---------------/,/^---------------/ { - /--------------- $threadId ---------------/b # skip first seperator line - /^---------------/s/.*// # replace sencond seperator line to empty line - p - }" - elif [ -n "$force" ]; then - local sed_script="/^Thread ${threadId}:/,/^$/p" - else - local sed_script="/nid=${threadId0x} /,/^$/p" - fi - sed "$sed_script" -n ${jstackFile} | tee ${append_file:+-a "$append_file"} - done -} - -headInfo() { - echo ================================================================================ - echo "$(date "+%Y-%m-%d %H:%M:%S.%N") [$((i+1))/$update_count]: ${COMMAND_LINE[@]}" - echo ================================================================================ - echo -} - -# if update_count <= 0, infinite loop till user interupted (eg: CTRL+C) -for ((i = 0; update_count <= 0 || i < update_count; ++i)); do - [ "$i" -gt 0 ] && sleep "$update_delay" - - [ -n "$append_file" ] && headInfo >> "$append_file" - [ "$update_count" -ne 1 ] && headInfo - - ps -Leo pid,lwp,user,comm,pcpu --no-headers | { - [ -z "${pid}" ] && - awk '$4=="java"{print $0}' || - awk -v "pid=${pid}" '$1==pid,$4=="java"{print $0}' - } | sort -k5 -r -n | head -n "${count}" | printStackOfThreads -done diff --git a/show-busy-java-threads.sh b/show-busy-java-threads.sh new file mode 120000 index 00000000..2c3d2d79 --- /dev/null +++ b/show-busy-java-threads.sh @@ -0,0 +1 @@ +show-busy-java-threads \ No newline at end of file diff --git a/show-duplicate-java-classes b/show-duplicate-java-classes index e17ff3ef..eeff8b8b 100755 --- a/show-duplicate-java-classes +++ b/show-duplicate-java-classes @@ -72,7 +72,7 @@ def print_class_paths(class_paths): print("class paths to find:") print("=" * 80) for idx, class_path in enumerate(class_paths): - print(("%-3d: %s" % (idx + 1, class_path))) + print("%-3d: %s" % (idx + 1, class_path)) if __name__ == '__main__': diff --git a/tcp-connection-state-counter b/tcp-connection-state-counter new file mode 100755 index 00000000..322dfabb --- /dev/null +++ b/tcp-connection-state-counter @@ -0,0 +1,23 @@ +#!/bin/bash +# @Function +# show count of tcp connection stat. +# +# @Usage +# $ ./tcp-connection-state-counter +# +# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#beer-tcp-connection-state-counter +# @author Jerry Lee (oldratlee at gmail dot com) +# @author @sunuslee (sunuslee at gmail dot com) + +# On MacOS, netstat need to using -p tcp to get only tcp output. +uname | grep Darwin -q && option_for_mac="-ptcp" + +netstat -tna $option_for_mac | awk 'NR > 2 { + s[$NF]++ +} + +END { + for(v in s) { + printf "%-11s %s\n", v, s[v] + } +}' | sort -nr -k2,2 diff --git a/tcp-connection-state-counter.sh b/tcp-connection-state-counter.sh deleted file mode 100755 index 51d29c15..00000000 --- a/tcp-connection-state-counter.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/bin/bash -# @Function -# show count of tcp connection stat. -# -# @Usage -# $ ./tcp-connection-state-counter.sh -# -# @online-doc https://github.com/oldratlee/useful-scripts/blob/master/docs/shell.md#beer-tcp-connection-state-countersh -# @author Jerry Lee (oldratlee at gmail dot com) -# @author @sunuslee (sunuslee at gmail dot com) - -# On MacOS, netstat need to using -p tcp to get only tcp output. -uname | grep Darwin -q && option_for_mac="-ptcp" - -netstat -tna $option_for_mac | awk 'NR > 2 { - s[$NF]++ -} - -END { - for(v in s) { - printf "%-11s %s\n", v, s[v] - } -}' | sort -nr -k2,2 diff --git a/tcp-connection-state-counter.sh b/tcp-connection-state-counter.sh new file mode 120000 index 00000000..60b5928f --- /dev/null +++ b/tcp-connection-state-counter.sh @@ -0,0 +1 @@ +tcp-connection-state-counter \ No newline at end of file