add retransform command tutorial

2024-04-21 10:21:39 +00:00 · 2021-01-13 20:01:30 +08:00
parent 65b025dba4
commit dd48b2b433
19 changed files with 635 additions and 0 deletions
@@ -587,6 +587,18 @@
                        cn: "command-jad-cn",
                    }
                },
+                {
+                    id: "command-mc-retransform",
+                    type: "COMMAND-CLASS",
+                    names: {
+                        en: "mc-retransform",
+                        cn: "mc-retransform",
+                    },
+                    ids: {
+                        en: "command-mc-retransform-en",
+                        cn: "command-mc-retransform-cn",
+                    }
+                },
                {
                    id: "command-mc-redefine",
                    type: "COMMAND-CLASS",
@@ -0,0 +1,16 @@
+
+
+
+
+在新的`Terminal 2`里，下载`arthas-boot.jar`，再用`java -jar`命令启动：
+
+`wget https://arthas.aliyun.com/arthas-boot.jar
+java -jar arthas-boot.jar`{{execute T2}}
+
+`arthas-boot`是`Arthas`的启动程序，它启动后，会列出所有的Java进程，用户可以选择需要诊断的目标进程。
+
+选择第一个进程，输入 `1`{{execute T2}} ，再`Enter/回车`：
+
+Attach成功之后，会打印Arthas LOGO。输入 `help`{{execute T2}} 可以获取到更多的帮助信息。
+
+![Arthas Boot](/arthas/scenarios/common-resources/assets/arthas-boot.png)
@@ -0,0 +1,95 @@
+下面介绍通过`jad`/`mc`/`retransform` 命令实现动态更新代码的功能。
+
+目前，访问 http://localhost/user/0 ，会返回500异常：
+
+`curl http://localhost/user/0`{{execute T3}}
+
+```
+{"timestamp":1550223186170,"status":500,"error":"Internal Server Error","exception":"java.lang.IllegalArgumentException","message":"id < 1","path":"/user/0"}
+```
+
+下面通过热更新代码，修改这个逻辑。
+
+### jad反编译UserController
+
+`jad --source-only com.example.demo.arthas.user.UserController > /tmp/UserController.java`{{execute T2}}
+
+jad反编译的结果保存在 `/tmp/UserController.java`文件里了。
+
+再打开一个`Terminal 3`，然后用vim来编辑`/tmp/UserController.java`：
+
+`vim /tmp/UserController.java`{{execute T3}}
+
+比如当 user id 小于1时，也正常返回，不抛出异常：
+
+```java
+    @GetMapping(value={"/user/{id}"})
+    public User findUserById(@PathVariable Integer id) {
+        logger.info("id: {}", (Object)id);
+        if (id != null && id < 1) {
+			return new User(id, "name" + id);
+            // throw new IllegalArgumentException("id < 1");
+        }
+        return new User(id.intValue(), "name" + id);
+    }
+```
+
+### sc查找加载UserController的ClassLoader
+
+`sc -d *UserController | grep classLoaderHash`{{execute T2}}
+
+```bash
+$ sc -d *UserController | grep classLoaderHash
+ classLoaderHash   1be6f5c3
+```
+
+可以发现是 spring boot `LaunchedURLClassLoader@1be6f5c3` 加载的。
+
+注意hashcode是变化的，需要先查看当前的ClassLoader信息，提取对应ClassLoader的hashcode。
+
+如果你使用`-c`，你需要手动输入hashcode：`-c <hashcode>`
+
+对于只有唯一实例的ClassLoader可以通过`--classLoaderClass`指定class name，使用起来更加方便.
+
+`--classLoaderClass` 的值是ClassLoader的类名，只有匹配到唯一的ClassLoader实例时才能工作，目的是方便输入通用命令，而`-c <hashcode>`是动态变化的。
+
+### mc
+
+保存好`/tmp/UserController.java`之后，使用`mc`(Memory Compiler)命令来编译，并且通过`--classLoaderClass`参数指定ClassLoader：
+
+`mc --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader /tmp/UserController.java -d /tmp`{{execute T2}}
+
+```bash
+$ mc --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader /tmp/UserController.java -d /tmp
+Memory compiler output:
+/tmp/com/example/demo/arthas/user/UserController.class
+Affect(row-cnt:1) cost in 346 ms
+```
+
+也可以通过`mc -c <classLoaderHash> /tmp/UserController.java -d /tmp`，使用`-c`参数指定ClassLoaderHash:
+
+```bash
+$ mc -c 1be6f5c3 /tmp/UserController.java -d /tmp
+```
+
+### retransform
+
+再使用`retransform`命令重新加载新编译好的`UserController.class`：
+
+`retransform /tmp/com/example/demo/arthas/user/UserController.class`{{execute T2}}
+
+```
+$ retransform /tmp/com/example/demo/arthas/user/UserController.class
+retransform success, size: 1
+```
+
+### 热修改代码结果
+
+`retransform`成功之后，再次访问 https://[[HOST_SUBDOMAIN]]-80-[[KATACODA_HOST]].environments.katacoda.com/user/0 ，结果是：
+
+```
+{
+  "id": 0,
+  "name": "name0"
+}
+```
@@ -0,0 +1,11 @@
+
+在“mc-retransform”中，我们演示了了Arthas的mc-retransform命令。如果有更多的技巧或者使用疑问，欢迎在Issue里提出。
+
+* Issues: https://github.com/alibaba/arthas/issues
+* 文档： https://arthas.aliyun.com/doc
+
+如果您在使用Arthas，请让我们知道。您的使用对我们非常重要：[查看](https://github.com/alibaba/arthas/issues/111)
+
+欢迎关注公众号，获取Arthas项目的信息、源码分析、案例实践。
+
+![Arthas公众号](/arthas/scenarios/common-resources/assets/qrcode_gongzhonghao.jpg)
@@ -0,0 +1,57 @@
+{
+  "title": "Arthas mc-retransform命令",
+  "description": "Arthas mc-retransform命令",
+  "difficulty": "精通者",
+  "time": "10-20 分钟",
+  "details": {
+    "steps": [
+      {
+        "title": "启动demo",
+        "text": "start-demo.md"
+      },
+      {
+        "title": "启动arthas-boot",
+        "text": "arthas-boot.md"
+      },
+      {
+        "title": "mc命令",
+        "text": "mc.md"
+      },
+      {
+        "title": "retransform命令",
+        "text": "retransform.md"
+      },
+      {
+        "title": "热更新代码",
+        "text": "case-jad-mc-retransform.md"
+      },
+      {
+        "title": "retransform命令更多说明",
+        "text": "retransform-more.md"
+      }
+    ],
+    "intro": {
+      "text": "intro.md"
+    },
+    "finish": {
+      "text": "finish.md"
+    },
+    "assets": {
+      "host01": []
+    }
+  },
+  "environment": {
+    "uilayout": "terminal",
+    "showdashboard": true,
+    "dashboards": [
+      {
+        "name": "Web Port 80",
+        "port": 80
+      }
+    ]
+  },
+  "backend": {
+    "imageid": "openjdk:15",
+    "environmentsprotocol": "http"
+  }
+}
@@ -0,0 +1,23 @@
+
+
+
+![Arthas](https://arthas.aliyun.com/doc/_images/arthas.png)
+
+`Arthas` 是Alibaba开源的Java诊断工具，深受开发者喜爱。在线排查问题，无需重启；动态跟踪Java代码；实时监控JVM状态。
+
+`Arthas` 支持JDK 6+，支持Linux/Mac/Windows，采用命令行交互模式，同时提供丰富的 `Tab` 自动补全功能，进一步方便进行问题的定位和诊断。
+
+当你遇到以下类似问题而束手无策时，Arthas可以帮助你解决：
+
+- 这个类从哪个 jar 包加载的？为什么会报各种类相关的 Exception？
+- 我改的代码为什么没有执行到？难道是我没 commit？分支搞错了？
+- 遇到问题无法在线上 debug，难道只能通过加日志再重新发布吗？
+- 线上遇到某个用户的数据处理有问题，但线上同样无法 debug，线下无法重现！
+- 是否有一个全局视角来查看系统的运行状况？
+- 有什么办法可以监控到JVM的实时运行状态？
+- 怎么快速定位应用的热点，生成火焰图？
+
+本教程会以一个普通的Spring Boot应用为例，演示mc-retransform命令。
+
+* Github: https://github.com/alibaba/arthas
+* 文档: https://arthas.aliyun.com/doc/
@@ -0,0 +1,7 @@
+
+> Memory Compiler/内存编译器，编译`.java`文件生成`.class`。
+
+
+可以通过`-c`/`--classLoaderClass`参数指定classloader，`-d`参数指定输出目录
+
+编译生成`.class`文件之后，可以结合`retransform`命令实现热更新代码。
@@ -0,0 +1,62 @@
+
+> 加载外部的`.class`文件，retransform jvm已加载的类。
+
+参考：[Instrumentation#retransformClasses](https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html#retransformClasses-java.lang.Class...-)
+
+
+### 查看 retransform entry
+
+`retransform -l`{{execute T2}}
+
+```bash
+$ retransform -l
+Id              ClassName       TransformCount  LoaderHash      LoaderClassName
+1               com.example.dem 1               null            null
+                o.arthas.user.U
+                serController
+```
+
+* TransformCount 统计在 ClassFileTransformer#transform 函数里尝试返回 entry对应的 .class文件的次数，但并不表明transform一定成功。
+
+### 删除指定 retransform entry
+
+`retransform -d 1`{{execute T2}}
+
+需要指定 id：
+
+```bash
+retransform -d 1
+```
+
+### 删除所有 retransform entry
+
+`retransform --deleteAll`{{execute T2}}
+
+```bash
+retransform --deleteAll
+```
+
+### 显式触发 retransform
+
+`retransform --classPattern com.example.demo.arthas.user.UserController`{{execute T2}}
+
+```bash
+$ retransform --classPattern com.example.demo.arthas.user.UserController
+retransform success, size: 1, classes:
+com.example.demo.arthas.user.UserController
+```
+
+> 注意：对于同一个类，当存在多个 retransform entry时，如果显式触发 retransform ，则最后添加的entry生效(id最大的)。
+
+### 消除 retransform 的影响
+
+如果对某个类执行 retransform 之后，想消除影响，则需要：
+
+* 删除这个类对应的 retransform entry
+* 重新触发 retransform
+
+> 如果不清除掉所有的 retransform entry，并重新触发 retransform ，则arthas stop时，retransform过的类仍然生效。
+
+在上面删掉 retransform entry，再显式触发 retransform之后，可以用 `jad`命令来确认之前retransform的结果已经被消除了。
+
+再次访问 https://[[HOST_SUBDOMAIN]]-80-[[KATACODA_HOST]].environments.katacoda.com/user/0 ，会抛出异常。
@@ -0,0 +1,19 @@
+
+> 加载外部的`.class`文件，retransform jvm已加载的类。
+
+参考：[Instrumentation#retransformClasses](https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html#retransformClasses-java.lang.Class...-)
+
+
+### 参数说明
+
+|参数名称|参数说明|
+|---:|:---|
+|[c:]|ClassLoader的hashcode|
+|`[classLoaderClass:]`|指定执行表达式的 ClassLoader 的 class name|
+|[p:]|外部的`.class`文件的完整路径，支持多个|
+
+
+### retransform的限制
+
+* 不允许新增加field/method
+* 正在跑的函数，没有退出不能生效。
@@ -0,0 +1,14 @@
+
+
+
+
+下载`demo-arthas-spring-boot.jar`，再用`java -jar`命令启动：
+
+`wget https://github.com/hengyunabc/spring-boot-inside/raw/master/demo-arthas-spring-boot/demo-arthas-spring-boot.jar
+java -jar demo-arthas-spring-boot.jar`{{execute T1}}
+
+`demo-arthas-spring-boot`是一个很简单的spring boot应用，源代码：[查看](https://github.com/hengyunabc/spring-boot-inside/tree/master/demo-arthas-spring-boot)
+
+启动之后，可以访问80端口： https://[[HOST_SUBDOMAIN]]-80-[[KATACODA_HOST]].environments.katacoda.com
+
+![Demo Web](/arthas/scenarios/common-resources/assets/demo-web.png)
@@ -0,0 +1,16 @@
+
+
+
+
+In the new `Terminal 2`, download `arthas-boot.jar` and start with the `java -jar` command:
+
+`wget https://arthas.aliyun.com/arthas-boot.jar
+java -jar arthas-boot.jar`{{execute T2}}
+
+`arthas-boot` is the launcher for `Arthas`. It lists all the Java processes, and the user can select the target process to be diagnosed.
+
+Select the first process, type `1`{{execute T2}} ，then type `Enter`：
+
+After the Attach is successful, Arthas LOGO is printed. Enter `help`{{execute T2}} for more help.
+
+![Arthas Boot](/arthas/scenarios/common-resources/assets/arthas-boot.png)
@@ -0,0 +1,98 @@
+This case introduces the ability to dynamically update code via the `jad`/`mc`/`retransform` command.
+
+Currently, visiting http://localhost/user/0 will return a 500 error:
+
+`curl http://localhost/user/0`{{execute T3}}
+
+```
+{"timestamp":1550223186170,"status":500,"error":"Internal Server Error","exception":"java.lang.IllegalArgumentException","message":"id < 1","path":"/user/0"}
+```
+
+This logic will be modified by `retransform` command below.
+
+## Use jad command to decompile UserController
+
+`jad --source-only com.example.demo.arthas.user.UserController > /tmp/UserController.java`{{execute T2}}
+
+The result of jad command will be saved in the `/tmp/UserController.java` file.
+
+
+Then open `Terminal 3`, use `vim` to edit `/tmp/UserController.java`:
+
+`vim /tmp/UserController.java`{{execute T3}}
+
+For example, when the user id is less than 1, it also returns normally without throwing an exception:
+
+```java
+    @GetMapping(value={"/user/{id}"})
+    public User findUserById(@PathVariable Integer id) {
+        logger.info("id: {}", (Object)id);
+        if (id != null && id < 1) {
+			return new User(id, "name" + id);
+            // throw new IllegalArgumentException("id < 1");
+        }
+        return new User(id.intValue(), "name" + id);
+    }
+```
+
+### Use sc command to find the ClassLoader that loads the UserController
+
+`sc -d *UserController | grep classLoaderHash`{{execute T2}}
+
+```bash
+$ sc -d *UserController | grep classLoaderHash
+ classLoaderHash   1be6f5c3
+```
+
+It can be found that it is loaded by spring boot `LaunchedURLClassLoader@1be6f5c3`.
+
+Note that the hashcode changes, you need to check the current ClassLoader information first, and extract the hashcode corresponding to the ClassLoader.
+
+if you use`-c`, you have to manually type hashcode by `-c <hashcode>`.
+
+For classloader with only one instance, it can be specified by `--classLoaderClass` using class name, which is more convenient to use.
+
+The value of `--classloaderclass` is the class name of classloader. It can only work when it matches a unique classloader instance. The purpose is to facilitate the input of general commands. However, `-c <hashcode>` is dynamic.
+
+### mc
+
+After saving `/tmp/UserController.java`, compile with the `mc` (Memory Compiler) command and specify the ClassLoader with the `--classLoaderClass` option:
+
+`mc --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader /tmp/UserController.java -d /tmp`{{execute T2}}
+
+```bash
+$ mc --classLoaderClass org.springframework.boot.loader.LaunchedURLClassLoader /tmp/UserController.java -d /tmp
+Memory compiler output:
+/tmp/com/example/demo/arthas/user/UserController.class
+Affect(row-cnt:1) cost in 346 ms
+```
+
+You can also execute `mc -c <classLoaderHash> /tmp/UserController.java -d /tmp`，using `-c` to specify ClassLoaderHash:
+
+```bash
+$ mc -c 1be6f5c3 /tmp/UserController.java -d /tmp
+```
+
+### retransform
+
+Then reload the newly compiled `UserController.class` with the `retransform` command:
+
+`retransform /tmp/com/example/demo/arthas/user/UserController.class`{{execute T2}}
+
+```
+$ retransform /tmp/com/example/demo/arthas/user/UserController.class
+retransform success, size: 1
+```
+
+### Check the results of the hotswap code
+
+After the `retransform` command is executed successfully, visit https://[[HOST_SUBDOMAIN]]-80-[[KATACODA_HOST]].environments.katacoda.com/user/0 again.
+
+The result is:
+
+```
+{
+  "id": 0,
+  "name": "name0"
+}
+```
@@ -0,0 +1,7 @@
+
+The `mc-retransform Tutorial` demonstrates the usage of mc-retransform. If you have more tips or questions, please feel free to tell or ask in Issue.
+
+* Issues: https://github.com/alibaba/arthas/issues
+* Documentation: https://arthas.aliyun.com/doc/en
+
+If you are using Arthas, please let us know that. Your use is very important to us: [View](https://github.com/alibaba/arthas/issues/111)
@@ -0,0 +1,57 @@
+{
+  "title": "Arthas mc-retransform Command",
+  "description": "Arthas mc-retransform Command ",
+  "difficulty": "master",
+  "time": "10-20 minutes",
+  "details": {
+    "steps": [
+      {
+        "title": "Start demo",
+        "text": "start-demo.md"
+      },
+      {
+        "title": "Start arthas-boot",
+        "text": "arthas-boot.md"
+      },
+      {
+        "title": "mc Command",
+        "text": "mc.md"
+      },
+      {
+        "title": "retransform Command",
+        "text": "retransform.md"
+      },
+      {
+        "title": "Hotswap Code",
+        "text": "case-jad-mc-retransform.md"
+      },
+      {
+        "title": "More information about retransform",
+        "text": "retransform-more.md"
+      }
+    ],
+    "intro": {
+      "text": "intro.md"
+    },
+    "finish": {
+      "text": "finish.md"
+    },
+    "assets": {
+      "host01": []
+    }
+  },
+  "environment": {
+    "uilayout": "terminal",
+    "showdashboard": true,
+    "dashboards": [
+      {
+        "name": "Web Port 80",
+        "port": 80
+      }
+    ]
+  },
+  "backend": {
+    "imageid": "openjdk:15",
+    "environmentsprotocol": "http"
+  }
+}
@@ -0,0 +1,39 @@
+
+
+
+![Arthas](https://arthas.aliyun.com/doc/_images/arthas.png)
+
+`Arthas` is a Java diagnostic tool open-sourced by Alibaba middleware team. Arthas helps developers in trouble-shooting issues in production environment for Java based applications without modifying code or restarting servers.
+
+`Arthas` supports JDK 6+, supports Linux/Mac/Windows.
+
+## Background
+
+Oftentimes the production system network is inaccessible from local development environment. If issues are encountered in production systems, it is impossible to use IDE to debug the application remotely. What’s even worse, debugging in production environment is unacceptable, as it will suspend all the threads, leading to services downtime.
+
+Developers could always try to reproduce the same issue on the test/staging environment. However, this is tricky as some issues cannot be reproduced easily in a different environment, or even disappear once restarted.
+
+And if you’re thinking of adding some logs to your code to help trouble-shoot the issue, you will have to go through the following lifecycle: test, staging, and then to production. Time is money! This approach is inefficient! Worse still, the issue may not be fixed since it might be irreproducible once the JVM is restarted, as described above.
+
+Arthas is built to solve these issues. A developer can troubleshoot production issues on the fly. No JVM restart, no additional code changes. Arthas works as an observer, that is, it will never suspend your running threads.
+
+## Key features
+
+- Check whether a class is loaded? Or where the class is loaded from? (Useful for trouble-shooting jar file conflicts)
+- Decompile a class to ensure the code is running as expected.
+- Check classloader statistics, e.g. the number of classloaders, the number of classes loaded per classloader, the classloader hierarchy, possible classloader leaks, etc.
+- Check the method invocation details, e.g. method parameter, returned values, exceptions and etc.
+- Check the stack trace of specified method invocation. This is useful when a developer wants to know the caller of the method.
+- Trace the method invocation to find slow sub-invocations.
+- Monitor method invocation statistics, e.g. QPS (Query Per Second), RT (Return Time), success rate and etc.
+- Monitor system metrics, thread states and CPU usage, GC statistics and etc.
+- Supports command line interactive mode, with auto-complete feature enabled.
+- Supports telnet and WebSocket, which enables both local and remote diagnostics with command line and browsers.
+- Supports profiler/Flame Graph
+- Supports JDK 6+
+- Supports Linux/Mac/Windows
+
+This tutorial takes a normal Spring Boot application as an example to demonstrate the the usage of mc-retransform.
+
+* Github: https://github.com/alibaba/arthas
+* Docs: https://arthas.aliyun.com/doc/en
@@ -0,0 +1,6 @@
+
+> Memory compiler, compiles `.java` files into `.class` files in memory.
+
+The classloader can be specified with the `-c`/`--classLoaderClass` option, the output directory can be specified with the `-d` option.
+
+After compiling the `.class` file, you can use the `retransform` command to update the loaded classes in JVM.
@@ -0,0 +1,64 @@
+
+> Load the external `*.class` files to retransform the loaded classes in JVM.
+
+Reference: [Instrumentation#retransformClasses](https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html#retransformClasses-java.lang.Class...-)
+
+
+### View retransform entry
+
+`retransform -l`{{execute T2}}
+
+```bash
+$ retransform -l
+Id              ClassName       TransformCount  LoaderHash      LoaderClassName
+1               com.example.dem 1               null            null
+                o.arthas.user.U
+                serController
+```
+
+* TransformCount counts the times of attempts to return the .class file corresponding to the entry in the ClassFileTransformer#transform method, but it does not mean that the transform must be successful.
+
+### Delete the specified retransform entry
+
+`retransform -d 1`{{execute T2}}
+
+Need to specify id:
+
+```bash
+retransform -d 1
+```
+
+### Delete all retransform entries
+
+`retransform --deleteAll`{{execute T2}}
+
+```bash
+retransform --deleteAll
+```
+
+### Explicitly trigger retransform
+
+`retransform --classPattern com.example.demo.arthas.user.UserController`{{execute T2}}
+
+```bash
+$ retransform --classPattern com.example.demo.arthas.user.UserController
+retransform success, size: 1, classes:
+com.example.demo.arthas.user.UserController
+```
+
+> Note: For the same class, when there are multiple retransform entries, if retransform is explicitly triggered, the entry added last will take effect (the one with the largest id).
+
+### Eliminate the influence of retransform
+
+If you want to eliminate the impact after performing retransform on a class, you need to:
+
+* Delete the retransform entry corresponding to this class
+* Re-trigger retransform
+
+> If you do not clear all retransform entries and trigger retransform again, the retransformed classes will still take effect when arthas stop.
+
+
+After deleting the retransform entry above and explicitly triggering the retransform, you can use the `jad` command to confirm that the result of the previous retransform has been eliminated.
+
+Visit https://[[HOST_SUBDOMAIN]]-80-[[KATACODA_HOST]].environments.katacoda.com/user/0 again, an exception will be thrown.
+
@@ -0,0 +1,18 @@
+
+> Load the external `*.class` files to retransform the loaded classes in JVM.
+
+Reference: [Instrumentation#retransformClasses](https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html#retransformClasses-java.lang.Class...-)
+
+
+### Options
+
+|Name|Specification|
+|---:|:---|
+|`[c:]`|hashcode of the class loader|
+|`[classLoaderClass:]`| The class name of the ClassLoader that executes the expression. |
+|`[p:]`|absolute path of the external `*.class`, multiple paths are separated with 'space'|
+
+### Restrictions of the retransform command
+
+* New field/method is not allowed
+* The function that is running, no exit can not take effect.
@@ -0,0 +1,14 @@
+
+
+
+
+Download `demo-arthas-spring-boot.jar`, and start with `java -jar` command:
+
+`wget https://github.com/hengyunabc/spring-boot-inside/raw/master/demo-arthas-spring-boot/demo-arthas-spring-boot.jar
+java -jar demo-arthas-spring-boot.jar`{{execute T1}}
+
+`demo-arthas-spring-boot` is a simple Spring Boot demo, the source code here: [View](https://github.com/hengyunabc/spring-boot-inside/tree/master/demo-arthas-spring-boot)
+
+After booting, access port 80: https://[[HOST_SUBDOMAIN]]-80-[[KATACODA_HOST]].environments.katacoda.com
+
+![Demo Web](/arthas/scenarios/common-resources/assets/demo-web.png)