[{"categories":["Java","Spring"],"collections":null,"content":"总结了迁移 JDK8 Spring MVC 5 项目到 JDK21 SpringBoot 3.5 的一些处理方式与步骤。 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:0:0","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#"},{"categories":["Java","Spring"],"collections":null,"content":"前置检查 检查与HTTP相关的第三方依赖和私有依赖，是否已经有支持Jakarta的版本，没有则需要克隆源码自行修改后发布到私有仓库。 检查项目中是否使用了Oracle相关的代码库，删除或使用其他开源库替代。 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:1:0","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#前置检查"},{"categories":["Java","Spring"],"collections":null,"content":"升级依赖 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:2:0","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#升级依赖"},{"categories":["Java","Spring"],"collections":null,"content":"修改版本控制 修改pom中的 parent 为 spring-boot-starter-parent，将大部分依赖版本号管理委托至 spring boot， 降低不同版本不兼容风险，例如： \u003cparent\u003e \u003cgroupId\u003eorg.springframework.boot\u003c/groupId\u003e \u003cartifactId\u003espring-boot-starter-parent\u003c/artifactId\u003e \u003cversion\u003e3.5.8\u003c/version\u003e \u003c/parent\u003e 添加版本控制后，需要将旧版本依赖的版本号全部删除，执行 maven dependency:resolve 查看哪些依赖版本号缺失， 并添加缺失的依赖版本号。 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:2:1","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#修改版本控制"},{"categories":["Java","Spring"],"collections":null,"content":"替换 spring-boot-starter 系列 一些依赖组件提供了原生的 spring-boot-starter，最好直接使用starter依赖替换旧的版本管理，降低迁移成本，例如： spring-boot-starter-web spring-boot-starter-test spring-boot-starter-validation mybatis-spring-boot-starter 其他的 starter 可以在官网查看 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:2:2","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#替换-spring-boot-starter-系列"},{"categories":["Java","Spring"],"collections":null,"content":"排除重复依赖 有一些无法升级或老旧的第三方包，可能兼容Java21，但自身会带来旧版本Spring或其他的依赖，此时使用命令 mvn dependency:tree -Dverbose检查依赖传递情况，找到重复依赖并排除，例如下面需要排除 commons-fileupload，转而使用 新版本 fileupload2： \u003cdependencyManagement\u003e \u003cdependency\u003e \u003cgroupId\u003ecom.bstek.ureport\u003c/groupId\u003e \u003cartifactId\u003eureport2-console\u003c/artifactId\u003e \u003cversion\u003e${ureport.version}\u003c/version\u003e \u003cexclusions\u003e \u003cexclusion\u003e \u003cgroupId\u003ecommons-fileupload\u003c/groupId\u003e \u003cartifactId\u003ecommons-fileupload\u003c/artifactId\u003e \u003c/exclusion\u003e \u003c/exclusions\u003e \u003c/dependency\u003e \u003cdependency\u003e \u003cgroupId\u003eorg.apache.commons\u003c/groupId\u003e \u003cartifactId\u003ecommons-fileupload2-jakarta-servlet6\u003c/artifactId\u003e \u003cversion\u003e${commons-fileupload.version}\u003c/version\u003e \u003c/dependency\u003e \u003c/dependencyManagement\u003e ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:2:3","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#排除重复依赖"},{"categories":["Java","Spring"],"collections":null,"content":"定制化停止更新的依赖 一些老旧的依赖可能在发布 Jakarta 前就已停更，此时需要获取到源码工程，修改源码兼容Java21和Spring Boot 3.5，并发布到私有仓库。 例如本次迁移中项目中使用Ureport2已经停更，且不支持Java21 Jakarta，修改源码大致步骤： Ureport源码中添加 spring-boot parent，删除由spring-boot管理的版本号。 修改或升级Ureport中的三方依赖。 修改源码以兼容新版的 Jakarta命名空间，操作同后续内容中的项目源码修改 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:2:4","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#定制化停止更新的依赖"},{"categories":["Java","Spring"],"collections":null,"content":"升级 maven 插件 一些maven插件在Java21环境下也需要升级，可以一并升级到最新版本，这个一般不会有兼容性问题。查看具体的版本传递可以使用插件 \u003cplugin\u003e \u003cgroupId\u003eorg.codehaus.mojo\u003c/groupId\u003e \u003cartifactId\u003eflatten-maven-plugin\u003c/artifactId\u003e \u003cversion\u003e1.7.3\u003c/version\u003e \u003cconfiguration\u003e \u003c/configuration\u003e \u003cexecutions\u003e \u003c!-- enable flattening --\u003e \u003cexecution\u003e \u003cid\u003eflatten\u003c/id\u003e \u003cphase\u003eprocess-resources\u003c/phase\u003e \u003cgoals\u003e \u003cgoal\u003eflatten\u003c/goal\u003e \u003c/goals\u003e \u003c/execution\u003e \u003c!-- ensure proper cleanup --\u003e \u003cexecution\u003e \u003cid\u003eflatten.clean\u003c/id\u003e \u003cgoals\u003e \u003cgoal\u003eclean\u003c/goal\u003e \u003c/goals\u003e \u003c/execution\u003e \u003c/executions\u003e \u003c/plugin\u003e ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:2:5","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#升级-maven-插件"},{"categories":["Java","Spring"],"collections":null,"content":"项目代码适配 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:3:0","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#项目代码适配"},{"categories":["Java","Spring"],"collections":null,"content":"javax 迁移 Jakarta 命名空间 使用IDEA自带工具批量替换 javax 命名空间为 jakarta，菜单位于 Refactoring -\u003e Migrate Packages and Classes -\u003e Java EE to Jakarta EE，本质上是替换代码文件中的字符串，可以新建自己的规则 批量替换其他命名空间。 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:3:1","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#javax-迁移-jakarta-命名空间"},{"categories":["Java","Spring"],"collections":null,"content":"迁移 spring xml 配置为代码配置方式 旧项目使用了spring xml 配置，需要将配置文件迁移为代码配置方式，并使用 @Configuration 注解声明为配置类，这个需要按项目 具体需求进行修改，注意最好不要使用xml与代码混合使用，建议全部迁移为代码配置方式。 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:3:2","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#迁移-spring-xml-配置为代码配置方式"},{"categories":["Java","Spring"],"collections":null,"content":"旧配置文件适配 spring boot 3 项目中配置文件为多个properties文件，使用springboot3中新引入的配置文件导入，由spring合并配置，例如在application.yaml中 引入旧项目配置文件base.properties、biz.properties等： spring: config: import: - classpath:base.properties - classpath:biz.properties 项目中如果使用了自定义的配置读取类，需要将配置文件注入到Spring Environment中，例如: public class MyPropertySourcePostProcessor implements EnvironmentPostProcessor { private static MapPropertySource mergeMyPropertySource() { Properties properties = Global.PROPERTIES_LOADER.getProperties(); Map\u003cString, Object\u003e map = new HashMap\u003c\u003e(); properties.forEach((k, v) -\u003e map.put(k.toString(), Global.getConfig(k.toString()))); MapPropertySource source = new MapPropertySource(\"myProperties\", map); return source; } @Override public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) { environment.getPropertySources() .addFirst(mergeMyPropertySource()); } } ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:3:3","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#旧配置文件适配-spring-boot-3"},{"categories":["Java","Spring"],"collections":null,"content":"编译测试 完成迁移后，需要编译测试项目，确保项目正常启动，并检查是否有兼容性问题。具体而言要完成回归测试、集成测试和性能测试， 同时部署后进行渗透漏洞扫描，确保当前依赖下无重大安全隐患。 ","date":"2025-12-15","objectID":"/posts/migrate-to-java21-springboot3/:4:0","tags":["Java","Spring"],"title":"Java8 Spring MVC 5 项目迁移 Java21 SpringBoot 3.5 记录","uri":"/posts/migrate-to-java21-springboot3/#编译测试"},{"categories":["Java"],"collections":null,"content":"记录一下JVM堆外内存泄露排查的过程 ","date":"2024-03-14","objectID":"/posts/jvm-native-memory-leak/:0:0","tags":["Java"],"title":"一次JVM堆外内存问题排查","uri":"/posts/jvm-native-memory-leak/#"},{"categories":["Java"],"collections":null,"content":"出现的问题 线上运行的Java应用在持续运行数小时后会因堆外内存不足而终止，查看日志可以看到类似输出： 先尝试了使用手动释放cached memory，发现还有大量内存未释放，基本排除了操作系统内存释放策略配置错误的问题 ","date":"2024-03-14","objectID":"/posts/jvm-native-memory-leak/:1:0","tags":["Java"],"title":"一次JVM堆外内存问题排查","uri":"/posts/jvm-native-memory-leak/#出现的问题"},{"categories":["Java"],"collections":null,"content":"开启JVM堆外内存追踪 在启动Java应用的参数中添加 -XX:NativeMemoryTracking=detail ，重启应用后运行一段时间，使用 jcmd $pid VM.native_memory summary 查看内存分配情况，可以看到类似输出 与 top 命令输出的 RSS 比对发现远小于其值，此时基本可以确定是 JNI 调用 Native Code 导致的堆外内存泄露 ","date":"2024-03-14","objectID":"/posts/jvm-native-memory-leak/:2:0","tags":["Java"],"title":"一次JVM堆外内存问题排查","uri":"/posts/jvm-native-memory-leak/#开启jvm堆外内存追踪"},{"categories":["Java"],"collections":null,"content":"查看未释放的堆外内存内容 先使用 pmp -x $pid 查看内存分布情况，对比前后两次的内容发现一直会申请一些64MB的内存块，并且始终处于未释放状态 查看 /proc/$pid/smaps 或 /proc/$pid/maps 找到对应内存起始和终止地址 使用 gdb -pid $pid attach 到问题进程， dump memory mem.bin 起始地址 终止地址 dump 出对应的多块内存块。 由于此次Java应用中存在中文字符，因此使用 strings -eS mem.bin 查看可显示的字符内容 经查看多个内存块后发现出现频率高的字符串没有明显规律，需要进一步排查 ","date":"2024-03-14","objectID":"/posts/jvm-native-memory-leak/:3:0","tags":["Java"],"title":"一次JVM堆外内存问题排查","uri":"/posts/jvm-native-memory-leak/#查看未释放的堆外内存内容"},{"categories":["Java"],"collections":null,"content":"监控内存分配 本次使用 Jemalloc 分析应用内存分配情况，首先在应用启动环境添加 export LD_PRELOAD=/usr/local/lib/libjemalloc.so export MALLOC_CONF=prof:true,lg_prof_interval:31,prof_prefix:/root/jeprof-result/app 重启应用后等待 Jemalloc 输出内存采样文件，查看多个采样文件后发现端倪 可以看到 524336 0.1% 100.0% 7386790 0.8% Java_java_util_zip_Inflater_init JNI调用 ","date":"2024-03-14","objectID":"/posts/jvm-native-memory-leak/:4:0","tags":["Java"],"title":"一次JVM堆外内存问题排查","uri":"/posts/jvm-native-memory-leak/#监控内存分配"},{"categories":["Java"],"collections":null,"content":"定位问题代码 使用 jstack $pid | grep Inflater 查找调用栈（可能需要多次） 发现在 Redis 序列化与反序列时使用了 JDK 中的 ZLIB 压缩算法，查看源码可以看到未调用 stream.close() infalter.end() 和 defalter.end()方法 顺便确认下 native code 调用 ","date":"2024-03-14","objectID":"/posts/jvm-native-memory-leak/:5:0","tags":["Java"],"title":"一次JVM堆外内存问题排查","uri":"/posts/jvm-native-memory-leak/#定位问题代码"},{"categories":["Java"],"collections":null,"content":"修改问题代码 直接看代码 ","date":"2024-03-14","objectID":"/posts/jvm-native-memory-leak/:6:0","tags":["Java"],"title":"一次JVM堆外内存问题排查","uri":"/posts/jvm-native-memory-leak/#修改问题代码"},{"categories":["分布式系统"],"collections":null,"content":"ELK 架构 文档 flowchart LR subgraph server-1 log-file-1 --\u003e filebeat-1 end subgraph server-2 log-file-2 --\u003e filebeat-2 end filebeat-1 -- push log --\u003e kafka filebeat-2 -- push log --\u003e kafka kafka -- sub log --\u003e logstash-filter logstash-filter -- pub log --\u003e kafka kafka -- sub log --\u003e logstash logstash -- push --\u003e es kibana -- query --\u003e es flowchart LR subgraph server-1 log-file-1 --\u003e filebeat-1 end subgraph server-2 log-file-2 --\u003e filebeat-2 end filebeat-1 -- push log --\u003e kafka filebeat-2 -- push log --\u003e kafka kafka -- sub log --\u003e logstash-filter logstash-filter -- pub log --\u003e kafka kafka -- sub log --\u003e logstash logstash -- push --\u003e es kibana -- query --\u003e es flowchart LR subgraph server-1 log-file-1 --\u003e filebeat-1 end subgraph server-2 log-file-2 --\u003e filebeat-2 end filebeat-1 -- push log --\u003e kafka filebeat-2 -- push log --\u003e kafka kafka -- sub log --\u003e logstash-filter logstash-filter -- pub log --\u003e kafka kafka -- sub log --\u003e logstash logstash -- push --\u003e es kibana -- query --\u003e es flowchart LR subgraph server-1 log-file-1 --\u003e filebeat-1 end subgraph server-2 log-file-2 --\u003e filebeat-2 end filebeat-1 -- push log --\u003e kafka filebeat-2 -- push log --\u003e kafka kafka -- sub log --\u003e logstash-filter logstash-filter -- pub log --\u003e kafka kafka -- sub log --\u003e logstash logstash -- push --\u003e es kibana -- query --\u003e es ","date":"2023-10-07","objectID":"/posts/elk%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E6%94%B6%E9%9B%86/:1:0","tags":["日志收集","分布式"],"title":"ELK分布式日志收集","uri":"/posts/elk%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E6%94%B6%E9%9B%86/#elk-架构"},{"categories":["分布式系统"],"collections":null,"content":"配置ES 修改文件config/elasticsearch.yml path.data path.logs network.host http.port 生成相应账户密码 ./bin/elasticsearch-reset-password -u elastic ./bin/elasticsearch-reset-password -u kibana-system ","date":"2023-10-07","objectID":"/posts/elk%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E6%94%B6%E9%9B%86/:2:0","tags":["日志收集","分布式"],"title":"ELK分布式日志收集","uri":"/posts/elk%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E6%94%B6%E9%9B%86/#配置es"},{"categories":["分布式系统"],"collections":null,"content":"配置Kibana server.port server.host elasticsearch.username elasticsearch.password i18n.locale ","date":"2023-10-07","objectID":"/posts/elk%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E6%94%B6%E9%9B%86/:3:0","tags":["日志收集","分布式"],"title":"ELK分布式日志收集","uri":"/posts/elk%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E6%94%B6%E9%9B%86/#配置kibana"},{"categories":["分布式系统"],"collections":null,"content":"配置Kafka 新建 config/kafka_server_jass.conf KafkaServer { org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-password\" user_admin=\"admin-password\" user_tom=\"tom-password\"; }; KafkaClient { org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-password\" user_admin=\"admin-password\" user_tom=\"tom-password\"; }; Client { org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-password\" user_admin=\"admin-password\"; }; Server { org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-password\" user_admin=\"admin-password\"; }; 修改 bin/kafka-server-start.sh, 最后一行前添加 export KAFKA_OPTS=\"-Djava.security.auth.login.config=$base_dir/../config/kafka_server_jaas.conf\" 新建 config/tom-client.conf sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule \\ required username=\"tom\" password=\"tom-password\"; security.protocol=SASL_PLAINTEXT sasl.mechanism=PLAIN 测试执行命令 ./bin/kafka-topics.sh --bootstrap-server ip:port --command-config config/tom-client.conf --list ","date":"2023-10-07","objectID":"/posts/elk%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E6%94%B6%E9%9B%86/:4:0","tags":["日志收集","分布式"],"title":"ELK分布式日志收集","uri":"/posts/elk%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E6%94%B6%E9%9B%86/#配置kafka"},{"categories":["算法"],"collections":null,"content":"字符串前缀相关操作 ","date":"2022-03-15","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%89%8D%E7%BC%80%E6%A0%91/:0:0","tags":["算法基础"],"title":"算法基础-前缀树","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%89%8D%E7%BC%80%E6%A0%91/#"},{"categories":["算法"],"collections":null,"content":"实现 package main type TrieNode struct { isWord bool children []*TrieNode } func NewTrieNode() *TrieNode { return \u0026TrieNode{ isWord: false, children: make([]*TrieNode, 26), } } type Trie struct { root *TrieNode } func NewTrie() Trie { return Trie{NewTrieNode()} } func (t *Trie) Insert(word string) { node := t.root for i := range word { idx := int(word[i] - 'a') if node.children[idx] == nil { node.children[idx] = NewTrieNode() } node = node.children[idx] } node.isWord = true } func (t *Trie) Search(word string) bool { node := t.GetNode(word) if node == nil || !node.isWord { return false } return true } func (t *Trie) StartsWith(prefix string) bool { return t.GetNode(prefix) != nil } func (t *Trie) GetNode(word string) *TrieNode { node := t.root for i := range word { idx := int(word[i] - 'a') child := node.children[idx] if child == nil { return nil } node = child } return node } func (t *Trie) Scan(prefix string) []string { node := t.GetNode(prefix) res := make([]string, 0) if node == nil { return res } var dfs func(node *TrieNode, path []byte) dfs = func(node *TrieNode, path []byte) { if node == nil { return } if node.isWord { res = append(res, prefix+string(path)) } for i := 0; i \u003c 26; i++ { child := node.children[i] if child != nil { path = append(path, byte(i+'a')) dfs(child, path) path = path[:len(path)-1] } } } dfs(node, []byte{}) return res } ","date":"2022-03-15","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%89%8D%E7%BC%80%E6%A0%91/:1:0","tags":["算法基础"],"title":"算法基础-前缀树","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%89%8D%E7%BC%80%E6%A0%91/#实现"},{"categories":["算法"],"collections":null,"content":"由图生成权重最小的树 给你一个points 数组，表示 2D 平面上的一些点，其中 points[i] = [xi, yi] 。 连接点 [xi, yi] 和点 [xj, yj] 的费用为它们之间的 曼哈顿距离 ：|xi - xj| + |yi - yj| ，其中 |val| 表示 val 的绝对值。 请你返回将所有点连接的最小总费用。只有任意两点之间 有且仅有 一条简单路径时，才认为所有点都已连接。 来源：力扣（LeetCode） 链接：https://leetcode-cn.com/problems/min-cost-to-connect-all-points 著作权归领扣网络所有。商业转载请联系官方授权，非商业转载请注明出处。 ","date":"2022-03-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%9C%80%E5%B0%8F%E7%94%9F%E6%88%90%E6%A0%91/:0:0","tags":["算法基础"],"title":"算法基础-最小生成树","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%9C%80%E5%B0%8F%E7%94%9F%E6%88%90%E6%A0%91/#"},{"categories":["算法"],"collections":null,"content":"Kruskal 所有边按权重排序，使用并查集选出不成环的边 func NewUF(n int) *UnionFind { parent := make([]int, n) for i := range parent { parent[i] = i } return \u0026UnionFind{ parent: parent, count: n, } } type UnionFind struct { parent []int count int } func (uf *UnionFind) Union(a, b int) { rootA := uf.Find(a) rootB := uf.Find(b) if rootA == rootB { return } uf.parent[rootA] = rootB uf.count-- } func (uf *UnionFind) Find(a int) int { for uf.parent[a] != a { uf.parent[a] = uf.parent[uf.parent[a]] a = uf.parent[a] } return a } func (uf *UnionFind) Connected(a, b int) bool { rootA := uf.Find(a) rootB := uf.Find(b) return rootA == rootB } func minCostConnectPoints(points [][]int) int { n := len(points) // graph [from, to, weight] graph := make([][]int, 0) for i := 0; i \u003c n; i++ { for j := i + 1; j \u003c n; j++ { edge := []int{i, j, dist(points, i, j)} graph = append(graph, edge) } } sort.Slice(graph, func(i, j int) bool { return graph[i][2] \u003c graph[j][2] }) uf := NewUF(n) res := 0 for _, edge := range graph { a, b, w := edge[0], edge[1], edge[2] if uf.Connected(a, b) { continue } uf.Union(a, b) res += w } return res } func abs(a int) int { if a \u003e 0 { return a } return -a } func dist(points [][]int, a, b int) int { return abs(points[a][0]-points[b][0]) + abs(points[a][1]-points[b][1]) } ","date":"2022-03-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%9C%80%E5%B0%8F%E7%94%9F%E6%88%90%E6%A0%91/:1:0","tags":["算法基础"],"title":"算法基础-最小生成树","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%9C%80%E5%B0%8F%E7%94%9F%E6%88%90%E6%A0%91/#kruskal"},{"categories":["算法"],"collections":null,"content":"Prim 图中每次切分，至少包含一条最小生成树的边（即权重最小的边必为最小生成树路径） func abs(a int) int { if a \u003e 0 { return a } return -a } func dist(points [][]int, a, b int) int { return abs(points[a][0]-points[b][0]) + abs(points[a][1]-points[b][1]) } type PriorityQueue struct { data [][]int } func (p *PriorityQueue) Len() int { return len(p.data) } func (p *PriorityQueue) Less(i, j int) bool { return p.data[i][1] \u003c p.data[j][1] } func (p *PriorityQueue) Swap(i, j int) { p.data[i], p.data[j] = p.data[j], p.data[i] } func (p *PriorityQueue) Push(x interface{}) { p.data = append(p.data, x.([]int)) } func (p *PriorityQueue) Pop() interface{} { v := p.data[len(p.data)-1] p.data = p.data[:len(p.data)-1] return v } func minCostConnectPoints(points [][]int) int { n := len(points) // graph[from] [to, weight] graph := make([][][]int, n) for i := 0; i \u003c n; i++ { // 邻接表表示的无向图，需要双向赋值 for j := 0; j \u003c n; j++ { if i == j { continue } graph[i] = append(graph[i], []int{j, dist(points, i, j)}) } } used := make([]bool, n) pq := \u0026PriorityQueue{[][]int{}} cut := func(a int) { for _, edge := range graph[a] { if used[edge[0]] { continue } heap.Push(pq, edge) } } used[0] = true cut(0) w := 0 for pq.Len() \u003e 0 { edge := heap.Pop(pq).([]int) if used[edge[0]] { continue } w += edge[1] used[edge[0]] = true cut(edge[0]) } return w } ","date":"2022-03-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%9C%80%E5%B0%8F%E7%94%9F%E6%88%90%E6%A0%91/:2:0","tags":["算法基础"],"title":"算法基础-最小生成树","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%9C%80%E5%B0%8F%E7%94%9F%E6%88%90%E6%A0%91/#prim"},{"categories":["算法"],"collections":null,"content":"用于将节点分为两组 ","date":"2022-03-08","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E4%BA%8C%E5%88%86%E5%9B%BE/:0:0","tags":["算法基础"],"title":"算法基础-二分图","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E4%BA%8C%E5%88%86%E5%9B%BE/#"},{"categories":["算法"],"collections":null,"content":"判断是否为二分图 ","date":"2022-03-08","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E4%BA%8C%E5%88%86%E5%9B%BE/:1:0","tags":["算法基础"],"title":"算法基础-二分图","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E4%BA%8C%E5%88%86%E5%9B%BE/#判断是否为二分图"},{"categories":["算法"],"collections":null,"content":"BFS 实现 func isBipartite(graph [][]int) bool { n := len(graph) visited := make([]bool, n) color := make([]bool, n) valid := true var bfs func(int) bfs = func(root int) { visited[root] = true queue := []int{root} for len(queue) \u003e 0 { node := queue[0] queue = queue[1:] for _, next := range graph[node] { if !visited[next] { color[next] = !color[node] visited[next] = true queue = append(queue, next) } else { if color[next] == color[node] { valid = false return } } } } } for i := 0; i \u003c n; i++ { if !visited[i] \u0026\u0026 valid { bfs(i) } } return valid } ","date":"2022-03-08","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E4%BA%8C%E5%88%86%E5%9B%BE/:1:1","tags":["算法基础"],"title":"算法基础-二分图","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E4%BA%8C%E5%88%86%E5%9B%BE/#bfs-实现"},{"categories":["算法"],"collections":null,"content":"DFS 实现 func possibleBipartition(n int, dislikes [][]int) bool { graph := make([][]int, n) for _, dislike := range dislikes { a, b := dislike[0]-1, dislike[1]-1 graph[a] = append(graph[a], b) graph[b] = append(graph[b], a) } valid := true visited := make([]bool, n) color := make([]bool, n) var dfs func(node int) dfs = func(node int) { if !valid { return } visited[node] = true for _, next := range graph[node] { if !visited[next] { color[next] = !color[node] dfs(next) } else { if color[next] == color[node] { valid = false return } } } } for i := 0; i \u003c n; i++ { if !visited[i] \u0026\u0026 valid { dfs(i) } } return valid } ","date":"2022-03-08","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E4%BA%8C%E5%88%86%E5%9B%BE/:1:2","tags":["算法基础"],"title":"算法基础-二分图","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E4%BA%8C%E5%88%86%E5%9B%BE/#dfs-实现"},{"categories":["算法"],"collections":null,"content":"用于判断连通性、等效等问题 ","date":"2022-03-08","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%B9%B6%E6%9F%A5%E9%9B%86/:0:0","tags":["算法基础"],"title":"算法基础-并查集","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%B9%B6%E6%9F%A5%E9%9B%86/#"},{"categories":["算法"],"collections":null,"content":"实现 func NewUF(n int) *UnionFind { parent := make([]int, n) for i := range parent { parent[i] = i } return \u0026UnionFind{ parent: parent, count: n, } } type UnionFind struct { parent []int count int } func (uf *UnionFind) Union(a, b int) { rootA := uf.Find(a) rootB := uf.Find(b) if rootA == rootB { return } uf.parent[rootA] = rootB uf.count-- } func (uf *UnionFind) Find(a int) int { for uf.parent[a] != a { // compress parent tree uf.parent[a] = uf.parent[uf.parent[a]] a = uf.parent[a] } return a } func (uf *UnionFind) Connected(a, b int) bool { rootA := uf.Find(a) rootB := uf.Find(b) return rootA == rootB } ","date":"2022-03-08","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%B9%B6%E6%9F%A5%E9%9B%86/:1:0","tags":["算法基础"],"title":"算法基础-并查集","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%B9%B6%E6%9F%A5%E9%9B%86/#实现"},{"categories":["算法"],"collections":null,"content":"经典排序算法 ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:0:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#"},{"categories":["算法"],"collections":null,"content":"冒泡排序 func sortBubble1(arr []int) []int { n := len(arr) // 有序区间 [0, i] for i := 0; i \u003c n; i++ { // 从最右侧两两交换，将无序区间中的最小值放置到i for j := n - 1; j \u003e i; j-- { if arr[j] \u003c arr[j-1] { swap(arr, j, j-1) } } } return arr } func sortBubble2(arr []int) { n := len(arr) exchanged := true // 有序区间 [0, i] for i := 0; i \u003c n \u0026\u0026 exchanged; i++ { // 当 exchanged 为 true 时，表明剩余无序区间已有序，结束排序 exchanged = false // 从最右侧两两交换，将无序区间中的最小值放置到i for j := n - 1; j \u003e i; j-- { if arr[j] \u003c arr[j-1] { exchanged = true swap(arr, j, j-1) } } } } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:1:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#冒泡排序"},{"categories":["算法"],"collections":null,"content":"快速排序 func sortQuick(arr []int) []int { return _sortQuick(arr, 0, len(arr) - 1) } func _sortQuick(arr []int, left, right int) []int { if left \u003c right { p := partition(arr, left, right) _sortQuick(arr, left, p-1) _sortQuick(arr, p+1, right) } return arr } func partition(arr []int, l int, r int) int { // l位空出 v := arr[l] for l \u003c r { for l \u003c r \u0026\u0026 arr[r] \u003e= v { r-- } // r位保存至l位，r位空出 arr[l] = arr[r] for l \u003c r \u0026\u0026 arr[l] \u003c= v { l++ } // l位保存至r位，l位空出 arr[r] = arr[l] } // 空出的l位填入基准值v arr[l] = v return l } class Solution { public int[] sortArray(int[] nums) { if (nums == null || nums.length \u003c= 1) { return nums; } quickSort(nums, 0, nums.length - 1); return nums; } private void quickSort(int[] nums, int l, int r) { if (l \u003e= r) { return; } int p = partition(nums, l, r); quickSort(nums, l, p - 1); quickSort(nums, p + 1, r); } private int partition(int[] nums, int l, int r) { int v = randomPiv(nums, l, r); while (l \u003c r) { while (l \u003c r \u0026\u0026 nums[r] \u003e= v) { r--; } nums[l] = nums[r]; while (l \u003c r \u0026\u0026 nums[l] \u003c= v) { l++; } nums[r] = nums[l]; } nums[l] = v; return l; } private int randomPiv(int[] nums, int l, int r) { int p = ThreadLocalRandom.current().nextInt(r - l + 1) + l; int temp = nums[l]; nums[l] = nums[p]; nums[p] = temp; return nums[l]; } } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:2:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#快速排序"},{"categories":["算法"],"collections":null,"content":"插入排序 func sortInsert(arr []int) []int{ // 有序区间 [0, i]，无序区间[i+1, +00] for i := 1; i \u003c len(arr); i++ { for j := i; j \u003e 0 \u0026\u0026 arr[j] \u003c arr[j-1]; j-- { // 小数前移 swap(arr, j, j - 1) } } return arr } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:3:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#插入排序"},{"categories":["算法"],"collections":null,"content":"希尔排序（插入排序改进版） 希尔为该算法发明人 func sortShell(arr []int) []int { for step := len(arr) / 2; step \u003e 0; step /= 2 { for i := step; i \u003c len(arr); i++ { for j := i; j \u003e= step \u0026\u0026 arr[j] \u003c arr[j-step]; j -= step { // 同组内小数前移 swap(arr, j, j-step) } } } return arr } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:4:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#希尔排序插入排序改进版"},{"categories":["算法"],"collections":null,"content":"选择排序 func sortSelect(arr []int) { for i := 0; i \u003c len(arr); i++ { m := i // 无序区间内选择最小值下标 m for j := i; j \u003c len(arr); j++ { if arr[j] \u003c arr[m] { m = j } } if m != i { swap(arr, i, m) } } } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:5:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#选择排序"},{"categories":["算法"],"collections":null,"content":"堆排序 func sortHeap(arr []int) []int { h := MinHeap{} h.build(arr) res := make([]int, 0, len(arr)) for len(h.heap) \u003e 0 { res = append(res, h.pop()) } return res } type MinHeap struct { heap []int } // 上浮操作：节点与大于节点值的父节点交换 func (h *MinHeap) up(k int) { if k == 0 { return } // 节点 k 比 父节点 pk 小时，交换两节点，实现节点上浮 pk := (k - 1) / 2 for k \u003e= 0 \u0026\u0026 h.heap[pk] \u003e h.heap[k] { h.heap[pk], h.heap[k] = h.heap[k], h.heap[pk] k = pk pk = (k - 1) / 2 } } // 下沉操作：节点与子节点中较小值交换 func (h *MinHeap) down(k int) { if len(h.heap) \u003c= 1 { return } lastNonLeaf := (len(h.heap) - 1 - 1) / 2 for k \u003c= lastNonLeaf { // 默认左侧节点为子节点中值最小节点 child := 2*k + 1 childV := h.heap[child] // 检查右侧节点，若比默认节点小，替换之 right := child + 1 if right \u003c len(h.heap) \u0026\u0026 h.heap[right] \u003c childV { child = right childV = h.heap[right] } // 若节点比子节点大，下沉 if h.heap[k] \u003e childV { h.heap[k], h.heap[child] = h.heap[child], h.heap[k] k = child } else { break } } } // 添加操作：加至末尾节点，后上浮尾节点 func (h *MinHeap) add(v int) { h.heap = append(h.heap, v) h.up(len(h.heap) - 1) } // 弹出操作：头节点弹出，尾节点放置头节点，后下沉头节点 func (h *MinHeap) pop() int { if len(h.heap) == 0 { panic(\"empty heap\") } res := h.heap[0] // swap last and first h.heap[0] = h.heap[len(h.heap)-1] // remove last h.heap = h.heap[:len(h.heap)-1] if len(h.heap) \u003e 0 { // re-heap h.down(0) } return res } // 构建树：从最后一个非叶子节点开始，执行下沉操作 func (h *MinHeap) build(arr []int) { h.heap = arr for i := (len(h.heap) - 2) / 2; i \u003e= 0; i-- { h.down(i) } } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:6:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#堆排序"},{"categories":["算法"],"collections":null,"content":"归并排序 func sortMerge(arr []int) []int { if len(arr) \u003c= 1 { return arr } mid := len(arr) / 2 return merge(sortMerge(arr[:mid]), sortMerge(arr[mid:])) } func merge(left, right []int) []int { res := make([]int, 0, len(left)+len(right)) // 每次处理一个元素 i, j := 0, 0 for i \u003c len(left) \u0026\u0026 j \u003c len(right) { if left[i] \u003c right[j] { res = append(res, left[i]) i++ } else { res = append(res, right[j]) j++ } } if i \u003c len(left) { res = append(res, left[i:]...) } if j \u003c len(right) { res = append(res, right[j:]...) } return res } class Solution { public int[] sortArray(int[] nums) { if (nums == null || nums.length \u003c= 1) { return nums; } sort(nums, 0, nums.length); return nums; } // [left, right) private void sort(int[] nums, int left, int right) { if (right - left \u003c= 1) { return; } int mid = left + (right - left) / 2; sort(nums, left, mid); sort(nums, mid, right); merge(nums, left, mid, right); } private void merge(int[] nums, int left, int mid, int right) { int[] arr = new int[right - left]; int p = 0; int i = left, j = mid; while (i \u003c mid \u0026\u0026 j \u003c right) { if (nums[i] \u003c nums[j]) { arr[p] = nums[i++]; } else { arr[p] = nums[j++]; } p++; } while (i \u003c mid) { arr[p++] = nums[i++]; } while (j \u003c right) { arr[p++] = nums[j++]; } System.arraycopy(arr, 0, nums, left, right-left); } } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:7:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#归并排序"},{"categories":["算法"],"collections":null,"content":"计数排序 func sortCount(arr []int) []int { // 建立排序桶 [min, max] minV, maxV := arr[0], arr[0] for _, v := range arr { if v \u003c minV { minV = v } if v \u003e maxV { maxV = v } } buckets := make([]int, maxV-minV+1) for _, v := range arr { // 需要 min 偏移 buckets[v-minV]++ } // 桶中提取数据 res := make([]int, 0, len(arr)) for i, count := range buckets { for ; count \u003e 0; count-- { // 加回偏移量 minV res = append(res, i+minV) } } return res } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:8:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#计数排序"},{"categories":["算法"],"collections":null,"content":"基数排序（桶排序的一种） 基数排序：根据键值的每位数字来分配桶 计数排序：每个桶只存储单一键值 桶排序：每个桶存储一定范围的数值，桶内使用其他排序算法 func sortRadix(arr []int) []int { // 默认 10 进制，故桶为 [0, 9] // 最大值位数默认 maxV := arr[0] for _, v := range arr { if v \u003e maxV { maxV = v } } maxLen := len(strconv.Itoa(maxV)) // 按低位至高位排序 step := 1 for i := 0; i \u003c maxLen; i++ { var buckets [10][]int // 插入排序桶 for _, v := range arr { bitV := v / step % 10 buckets[bitV] = append(buckets[bitV], v) } // 更新数组 idx := 0 for _, bucket := range buckets { for _, v := range bucket { arr[idx] = v idx++ } } step *= 10 } return arr } ","date":"2021-09-10","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/:9:0","tags":["算法基础"],"title":"算法基础-排序","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E6%8E%92%E5%BA%8F/#基数排序桶排序的一种"},{"categories":["缓存"],"collections":null,"content":"Redis集群相关 ","date":"2021-08-16","objectID":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/:0:0","tags":["Redis"],"title":"Redis主从与集群模式","uri":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/#"},{"categories":["缓存"],"collections":null,"content":"主从模式 主节点Redis配置 redis-6090.conf： port 6090 protected-mode no 从节点Redis配置 redis-6091.conf： port 6091 protected-mode no slaveof 127.0.0.1 6090 从节点Redis配置 redis-6092.conf： port 6092 protected-mode no slaveof 127.0.0.1 6090 Redis Sentinel配置模版 redis-sentinel-cluster.tmpl： port ${PORT} sentinel monitor mymaster 127.0.0.1 6090 2 sentinel down-after-milliseconds mymaster 5000 sentinel failover-timeout mymaster 60000 Redis Sentinel 实例的配置文件 redis-sentinel-cluster-config.sh，生成 5000-5002 3个实例： #!/bin/bash for port in `seq 5000 5002`; do \\ mkdir -p ./redis-sentinel-cluster/${port}/conf PORT=${port} envsubst \u003c ./redis-sentinel-cluster.tmpl \u003e ./redis-sentinel-cluster/${port}/conf/sentinel.conf chmod 777 ./redis-sentinel-cluster/${port}/conf/sentinel.conf mkdir -p ./redis-sentinel-cluster/${port}/data done 使用 Docker Compose 部署： version: \"3\" services: redis7000: image: redis:alpine container_name: redis6090 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-6090/conf:/usr/local/etc/redis - ./redis-6090/data:/data redis7001: image: redis:alpine container_name: redis6091 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-6091/conf:/usr/local/etc/redis - ./redis-6091/data:/data redis7002: image: redis:alpine container_name: redis6092 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-6092/conf:/usr/local/etc/redis - ./redis-6092/data:/data sentinel5000: image: redis:alpine container_name: sentinel5000 command: \"redis-server /usr/local/etc/redis/sentinel.conf --sentinel\" network_mode: host volumes: - ./redis-sentinel-cluster/5000/conf:/usr/local/etc/redis - ./redis-sentinel-cluster/5000/data:/data sentinel5001: image: redis:alpine container_name: sentinel5001 command: \"redis-server /usr/local/etc/redis/sentinel.conf --sentinel\" network_mode: host volumes: - ./redis-sentinel-cluster/5001/conf:/usr/local/etc/redis - ./redis-sentinel-cluster/5001/data:/data sentinel5002: image: redis:alpine container_name: sentinel5002 command: \"redis-server /usr/local/etc/redis/sentinel.conf --sentinel\" network_mode: host volumes: - ./redis-sentinel-cluster/5002/conf:/usr/local/etc/redis - ./redis-sentinel-cluster/5002/data:/data ","date":"2021-08-16","objectID":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/:1:0","tags":["Redis"],"title":"Redis主从与集群模式","uri":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/#主从模式"},{"categories":["缓存"],"collections":null,"content":"Cluster 模式 参考资料 ","date":"2021-08-16","objectID":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/:2:0","tags":["Redis"],"title":"Redis主从与集群模式","uri":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/#cluster-模式"},{"categories":["缓存"],"collections":null,"content":"Redis集群配置文件 Redis配置模版 redis-cluster.tmpl： port ${PORT} protected-mode no cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 5000 appendonly yes 各个Redis实例配置生成脚本 redis-cluster-config.sh，脚本生成端口号 7000-7005 6个Redis实例： #!/bin/bash for port in `seq 7000 7005`; do mkdir -p ./redis-cluster/${port}/conf PORT=${port} envsubst \u003c ./redis-cluster.tmpl \u003e ./redis-cluster/${port}/conf/redis.conf chmod 666 ./redis-cluster/${port}/conf/redis.conf mkdir -p ./redis-cluster/${port}/data done ","date":"2021-08-16","objectID":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/:2:1","tags":["Redis"],"title":"Redis主从与集群模式","uri":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/#redis集群配置文件"},{"categories":["缓存"],"collections":null,"content":"Redis集群启动 使用 Docker Compose 部署： version: \"3\" services: redis7000: image: redis:alpine container_name: redis7000 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-cluster/7000/conf:/usr/local/etc/redis - ./redis-cluster/7000/data:/data redis7001: image: redis:alpine container_name: redis7001 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-cluster/7001/conf:/usr/local/etc/redis - ./redis-cluster/7001/data:/data redis7002: image: redis:alpine container_name: redis7002 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-cluster/7002/conf:/usr/local/etc/redis - ./redis-cluster/7002/data:/data redis7003: image: redis:alpine container_name: redis7003 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-cluster/7003/conf:/usr/local/etc/redis - ./redis-cluster/7003/data:/data redis7004: image: redis:alpine container_name: redis7004 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-cluster/7004/conf:/usr/local/etc/redis - ./redis-cluster/7004/data:/data redis7005: image: redis:alpine container_name: redis7005 command: \"redis-server /usr/local/etc/redis/redis.conf\" network_mode: host volumes: - ./redis-cluster/7005/conf:/usr/local/etc/redis - ./redis-cluster/7005/data:/data 容器启动完毕后，连接至任意一个Redis示例，开始初始化集群： redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 \\ 127.0.0.1:7002 127.0.0.1:7003 \\ 127.0.0.1:7004 127.0.0.1:7005 \\ --cluster-replicas 1 执行完毕后将构建一个3主3从的Redis集群。 ","date":"2021-08-16","objectID":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/:2:2","tags":["Redis"],"title":"Redis主从与集群模式","uri":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/#redis集群启动"},{"categories":["缓存"],"collections":null,"content":"测试集群 对于Redis集群，命令行模式需要增加 -c 选项，否则无法操作Redis，即 redis-cli -c -p 7000 连接到端口号为7000的Redis，查看节点信息： 127.0.0.1:7000\u003e cluster nodes output\u003e 2fc17db8bb8818ec8ac50f40f07bfbdf1ee8d56c 127.0.0.1:7002@17002 master - 0 1629082981000 3 connected 10923-16383 fc92c1709a88bdca8e86b9b5a27d035609e46059 127.0.0.1:7001@17001 master - 0 1629082982000 2 connected 5461-10922 3bd1b8cbe2e0e30a77dfa2b92dd70ed834bfcba4 127.0.0.1:7000@17000 myself,master - 0 1629082981000 1 connected 0-5460 cf361fc2b72ca2bfd070b282754ad19da7640485 127.0.0.1:7003@17003 slave 3bd1b8cbe2e0e30a77dfa2b92dd70ed834bfcba4 0 1629082982590 1 connected fddc21542112cdcd8857bbfc77481b673e3a543e 127.0.0.1:7004@17004 slave fc92c1709a88bdca8e86b9b5a27d035609e46059 0 1629082981586 2 connected 2a5e09014c852fd15c2e54d500c532ec6ee2490e 127.0.0.1:7005@17005 slave 2fc17db8bb8818ec8ac50f40f07bfbdf1ee8d56c 0 1629082982088 3 connected 现在让7000的实例离线30秒： redis-cli -p 7000 DEBUG sleep 30 现在看一下集群信息，可以看到主节点已经由7000自动切换到了7003，： 127.0.0.1:7000\u003e cluster nodes 2fc17db8bb8818ec8ac50f40f07bfbdf1ee8d56c 127.0.0.1:7002@17002 master - 0 1629083362653 3 connected 10923-16383 fc92c1709a88bdca8e86b9b5a27d035609e46059 127.0.0.1:7001@17001 master - 0 1629083361000 2 connected 5461-10922 3bd1b8cbe2e0e30a77dfa2b92dd70ed834bfcba4 127.0.0.1:7000@17000 myself,slave cf361fc2b72ca2bfd070b282754ad19da7640485 0 1629083362000 7 connected cf361fc2b72ca2bfd070b282754ad19da7640485 127.0.0.1:7003@17003 master - 0 1629083362000 7 connected 0-5460 fddc21542112cdcd8857bbfc77481b673e3a543e 127.0.0.1:7004@17004 slave fc92c1709a88bdca8e86b9b5a27d035609e46059 0 1629083362553 2 connected 2a5e09014c852fd15c2e54d500c532ec6ee2490e 127.0.0.1:7005@17005 slave 2fc17db8bb8818ec8ac50f40f07bfbdf1ee8d56c 0 1629083361647 3 connected 可以看到7000已经切换为了salve模式，7003切换成了master模式。 使用Go语言测试读写： package main import ( \"context\" \"fmt\" \"github.com/go-redis/redis/v8\" ) var ctx = context.Background() func redisAddrList() []string { ip := \"localhost\" addrList := []string{} for port := 7000; port \u003c 7006; port++ { addrList = append(addrList, fmt.Sprintf(\"%s:%d\", ip, port)) } return addrList } func NewRedisClient() *redis.ClusterClient { return redis.NewClusterClient(\u0026redis.ClusterOptions{ Addrs: redisAddrList(), }) } func main() { rdb := NewRedisClient() err := rdb.Set(ctx, \"foo\", \"bar\", 0).Err() if err != nil { panic(err) } res, err := rdb.Get(ctx, \"foo\").Result() if err != nil { panic(err) } fmt.Println(res) } ","date":"2021-08-16","objectID":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/:2:3","tags":["Redis"],"title":"Redis主从与集群模式","uri":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/#测试集群"},{"categories":["缓存"],"collections":null,"content":"扩充节点 当前节点和槽分配情况： ~/docker-app/redis-cluster » redis-cli -p 7000 cluster nodes 9a353c214cb677b1e996573b4550875fb0754b65 127.0.0.1:7000@17000 myself,slave 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 0 1736083233000 7 connected 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 127.0.0.1:7003@17003 master - 0 1736083234560 7 connected 0-5460 3b8b46eac23b396e1d0b1adcfb70d086ee539da5 127.0.0.1:7004@17004 slave bd393aeacf7de92c8a3816cbfb1c497a310a418b 0 1736083235665 2 connected bd393aeacf7de92c8a3816cbfb1c497a310a418b 127.0.0.1:7001@17001 master - 0 1736083235000 2 connected 5461-10922 3c21e101ea340698769de95812a98275693d3b2d 127.0.0.1:7002@17002 slave 57011e62426c833268c7f01b248b0e8f44af443a 0 1736083234660 8 connected 57011e62426c833268c7f01b248b0e8f44af443a 127.0.0.1:7005@17005 master - 0 1736083235563 8 connected 10923-16383 在工作目录下复制一个redis配置 cp -r 7005 7006，并将其端口号配置为7006，使用docker启动临时redis docker run -d \\ --name redis7006 \\ --network host \\ -v ./redis-cluster/7006/conf:/usr/local/etc/redis \\ -v ./redis-cluster/7006/data:/data \\ redis:6.2 redis-server /usr/local/etc/redis/redis.conf 确认启动成功后，执行命令添加新的 redis node 到集群中去： ~/docker-app/redis-cluster » redis-cli --cluster add-node 127.0.0.1:7006 127.0.0.1:7000 ing@ing-pc \u003e\u003e\u003e Adding node 127.0.0.1:7006 to cluster 127.0.0.1:7000 \u003e\u003e\u003e Performing Cluster Check (using node 127.0.0.1:7000) S: 9a353c214cb677b1e996573b4550875fb0754b65 127.0.0.1:7000 slots: (0 slots) slave replicates 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 M: 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 127.0.0.1:7003 slots:[0-5460] (5461 slots) master 1 additional replica(s) S: 3b8b46eac23b396e1d0b1adcfb70d086ee539da5 127.0.0.1:7004 slots: (0 slots) slave replicates bd393aeacf7de92c8a3816cbfb1c497a310a418b M: bd393aeacf7de92c8a3816cbfb1c497a310a418b 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 3c21e101ea340698769de95812a98275693d3b2d 127.0.0.1:7002 slots: (0 slots) slave replicates 57011e62426c833268c7f01b248b0e8f44af443a M: 57011e62426c833268c7f01b248b0e8f44af443a 127.0.0.1:7005 slots:[10923-16383] (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. \u003e\u003e\u003e Check for open slots... \u003e\u003e\u003e Check slots coverage... [OK] All 16384 slots covered. \u003e\u003e\u003e Send CLUSTER MEET to node 127.0.0.1:7006 to make it join the cluster. [OK] New node added correctly. 此时再次查看节点信息，可以看到7006已经加入到cluster中，： ~/docker-app/redis-cluster » redis-cli -p 7000 cluster nodes ing@ing-pc 9a353c214cb677b1e996573b4550875fb0754b65 127.0.0.1:7000@17000 myself,slave 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 0 1736083995000 7 connected 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 127.0.0.1:7003@17003 master - 0 1736083997000 7 connected 0-5460 3b8b46eac23b396e1d0b1adcfb70d086ee539da5 127.0.0.1:7004@17004 slave bd393aeacf7de92c8a3816cbfb1c497a310a418b 0 1736083995380 2 connected 127.0.0.1:7006@17006 master - 0 1736083996384 0 connected bd393aeacf7de92c8a3816cbfb1c497a310a418b 127.0.0.1:7001@17001 master - 0 1736083997390 2 connected 5461-10922 3c21e101ea340698769de95812a98275693d3b2d 127.0.0.1:7002@17002 slave 57011e62426c833268c7f01b248b0e8f44af443a 0 1736083996585 8 connected 57011e62426c833268c7f01b248b0e8f44af443a 127.0.0.1:7005@17005 master - 0 1736083996000 8 connected 10923-16383 如果需要给7006指定一个slave，则启动7007端口的redis后，执行命令以下命令： $ redis-cli -p 7000 --cluster add-node 127.0.0.1:7007 127.0.0.1:7000 --cluster-slave --cluster-master-id d1c7e05fefad29888e1e4eb3a19574b94a280932 其中 –cluster-slave 指明新节点为slave，d1c7e05fefad29888e1e4eb3a19574b94a280932是7007节点id 新加入的节点是没有自动分配slot的，可以查看确认： ~/docker-app/redis-cluster » redis-cli -p 7000 -c cluster slots 1 ↵ ing@ing-pc 1) 1) (integer) 0 2) (integer) 5460 3) 1) \"127.0.0.1\" 2) (integer) 7003 3) \"0a50949e09aa0ebce0dbad1c8b0f301adbd62780\" 4) 1) \"127.0.0.1\" 2) (integer) 7000 3) \"9a353c214cb677b1e996573b4550875fb0754b65\" 2) 1) (integer) 5461 2) (integer) 10922 3) 1) \"127.0.0.1\" 2) (integer) 7001 3) \"bd393aeacf7de92c8a3816cbfb1c497a310a418b\" 4) 1) \"127.0.0.1\" 2) (integer) 7004 3) \"3b8b46eac23b3","date":"2021-08-16","objectID":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/:2:4","tags":["Redis"],"title":"Redis主从与集群模式","uri":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/#扩充节点"},{"categories":["缓存"],"collections":null,"content":"删除节点 停止7006容器 docker stop 7006，使用CLUSTER FORGET移除节点： ~/docker-app/redis-cluster » redis-cli -p 7000 -c CLUSTER FORGET d1c7e05fefad29888e1e4eb3a19574b94a280932 ing@ing-pc OK ~/docker-app/redis-cluster » redis-cli -p 7000 -c CLUSTER NODES ing@ing-pc 9a353c214cb677b1e996573b4550875fb0754b65 127.0.0.1:7000@17000 myself,slave 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 0 1736085038000 7 connected 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 127.0.0.1:7003@17003 master - 0 1736085039571 7 connected 1365-5460 3b8b46eac23b396e1d0b1adcfb70d086ee539da5 127.0.0.1:7004@17004 slave bd393aeacf7de92c8a3816cbfb1c497a310a418b 0 1736085039673 2 connected bd393aeacf7de92c8a3816cbfb1c497a310a418b 127.0.0.1:7001@17001 master - 0 1736085038667 2 connected 6827-10922 3c21e101ea340698769de95812a98275693d3b2d 127.0.0.1:7002@17002 slave 57011e62426c833268c7f01b248b0e8f44af443a 0 1736085039068 8 connected 57011e62426c833268c7f01b248b0e8f44af443a 127.0.0.1:7005@17005 master - 0 1736085039000 8 connected 12288-16383 ~/docker-app/redis-cluster » redis-cli -p 7000 -c CLUSTER SLOTS ing@ing-pc 1) 1) (integer) 1365 2) (integer) 5460 3) 1) \"127.0.0.1\" 2) (integer) 7003 3) \"0a50949e09aa0ebce0dbad1c8b0f301adbd62780\" 4) 1) \"127.0.0.1\" 2) (integer) 7000 3) \"9a353c214cb677b1e996573b4550875fb0754b65\" 2) 1) (integer) 6827 2) (integer) 10922 3) 1) \"127.0.0.1\" 2) (integer) 7001 3) \"bd393aeacf7de92c8a3816cbfb1c497a310a418b\" 4) 1) \"127.0.0.1\" 2) (integer) 7004 3) \"3b8b46eac23b396e1d0b1adcfb70d086ee539da5\" 3) 1) (integer) 12288 2) (integer) 16383 3) 1) \"127.0.0.1\" 2) (integer) 7005 3) \"57011e62426c833268c7f01b248b0e8f44af443a\" 4) 1) \"127.0.0.1\" 2) (integer) 7002 3) \"3c21e101ea340698769de95812a98275693d3b2d\" 发现此时slot缺失了一部分，查看cluster状态会发现集群此时处于 fail ~/docker-app/redis-cluster » redis-cli -p 7000 -c CLUSTER INFO ing@ing-pc cluster_state:fail cluster_slots_assigned:12288 cluster_slots_ok:12288 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:9 cluster_my_epoch:7 cluster_stats_messages_ping_sent:7170 cluster_stats_messages_pong_sent:7271 cluster_stats_messages_update_sent:2 cluster_stats_messages_sent:14443 cluster_stats_messages_ping_received:7270 cluster_stats_messages_pong_received:11255 cluster_stats_messages_meet_received:1 cluster_stats_messages_fail_received:3 cluster_stats_messages_auth-req_received:1 cluster_stats_messages_received:18530 原因是7006在持有slot的状态下被移除了集群，此时我们需要再次reshard 当然你可以设置 cluster-require-full-coverage=no 让集群强制在缺失slot的情况下继续提供服务 执行命令修复集群 ~/docker-app/redis-cluster » redis-cli --cluster fix 127.0.0.1:7000 1 ↵ ing@ing-pc 127.0.0.1:7003 (0a50949e...) -\u003e 0 keys | 4096 slots | 1 slaves. 127.0.0.1:7001 (bd393aea...) -\u003e 0 keys | 4096 slots | 1 slaves. 127.0.0.1:7005 (57011e62...) -\u003e 1 keys | 4096 slots | 1 slaves. [OK] 1 keys in 3 masters. 0.00 keys per slot on average. \u003e\u003e\u003e Performing Cluster Check (using node 127.0.0.1:7000) S: 9a353c214cb677b1e996573b4550875fb0754b65 127.0.0.1:7000 slots: (0 slots) slave replicates 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 M: 0a50949e09aa0ebce0dbad1c8b0f301adbd62780 127.0.0.1:7003 slots:[1365-5460] (4096 slots) master 1 additional replica(s) S: 3b8b46eac23b396e1d0b1adcfb70d086ee539da5 127.0.0.1:7004 slots: (0 slots) slave replicates bd393aeacf7de92c8a3816cbfb1c497a310a418b M: bd393aeacf7de92c8a3816cbfb1c497a310a418b 127.0.0.1:7001 slots:[6827-10922] (4096 slots) master 1 additional replica(s) S: 3c21e101ea340698769de95812a98275693d3b2d 127.0.0.1:7002 slots: (0 slots) slave replicates 57011e62426c833268c7f01b248b0e8f44af443a M: 57011e62426c833268c7f01b248b0e8f44af443a 127.0.0.1:7005 slots:[12288-16383] (4096 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. \u003e\u003e\u003e Check for open slots... \u003e\u003e\u003e Check slots coverage... [ERR] Not all 16384 slots are covered by nodes. \u003e\u003e\u003e Fixing slots coverage... The following uncovered slots have no keys across the cluster: [0-1364],[5461-6826],[10923-12287] Fix these slots by covering with a random node? (type 'yes' to acce","date":"2021-08-16","objectID":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/:2:5","tags":["Redis"],"title":"Redis主从与集群模式","uri":"/posts/redis%E4%B8%BB%E4%BB%8E%E4%B8%8E%E9%9B%86%E7%BE%A4%E6%A8%A1%E5%BC%8F/#删除节点"},{"categories":["算法"],"collections":null,"content":"堆，亦称为优先队列 ","date":"2021-08-13","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%A0%86/:0:0","tags":["算法基础"],"title":"算法基础-堆","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%A0%86/#"},{"categories":["算法"],"collections":null,"content":"最小堆 设堆大小为 size，对于下标为k的节点： 父节点下标为 (k - 1) / 2 左子节点下标为 2 * k + 1，右子节点下标为 2 * k + 2 最后一个非叶子节点的下标为 (size - 2) / 2 (即最后一个节点 size - 1的父节点 ((size - 1) - 1) / 2) go语言参考实现： type MinHeap struct { heap []int } func NewMinHeap() *MinHeap { return \u0026MinHeap{heap: []int{}} } func (h *MinHeap) up(k int) { if k == 0 { return } for pk := (k - 1) / 2; k \u003e= 0 \u0026\u0026 h.heap[pk] \u003e h.heap[k]; { h.heap[pk], h.heap[k] = h.heap[k], h.heap[pk] k = pk pk = (k - 1) / 2 } } func (h *MinHeap) down(k int) { if len(h.heap) == 1 { return } last := (len(h.heap) - 2) / 2 for k \u003c= last { child := 2*k + 1 // left child childV := h.heap[child] right := child + 1 if right \u003c len(h.heap) \u0026\u0026 h.heap[right] \u003c childV { child = right childV = h.heap[right] } if h.heap[k] \u003e childV { h.heap[k], h.heap[child] = h.heap[child], h.heap[k] k = child } else { break } } } func (h *MinHeap) add(v int) { h.heap = append(h.heap, v) h.up(len(h.heap) - 1) } func (h *MinHeap) pop() (int, bool) { if len(h.heap) == 0 { return 0, false } res := h.heap[0] h.heap[0] = h.heap[len(h.heap)-1] h.heap = h.heap[:len(h.heap)-1] if len(h.heap) \u003e 0 { h.down(0) } return res, true } func (h *MinHeap) heapify(arr []int) { h.heap = arr for i := (len(h.heap) - 2) / 2; i \u003e= 0; i-- { h.down(i) } } ","date":"2021-08-13","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%A0%86/:1:0","tags":["算法基础"],"title":"算法基础-堆","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E5%A0%86/#最小堆"},{"categories":["数据库"],"collections":null,"content":"分布式事务简介 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:0:0","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#"},{"categories":["数据库"],"collections":null,"content":"理论基础 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:1:0","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#理论基础"},{"categories":["数据库"],"collections":null,"content":"2PC 1.prepare ┌─────────┐ 1.prepare ┌─────────────┤ ├────────────┐ │ 2.ok/fail │ TM │ 2.ok/fail │ │ ┌────────►│ │◄───────┐ │ │ │ └─┬────┬──┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ 3.commit│ │ 3.commit │ │ │ │ /rollback │ /roolback│ │ │ │ │ │ │ │ │ │ │ │ │ │ ▼ │ │ │ │ ▼ ┌────┴───┐ │ │ ┌─────┴────┐ │ │◄──────┘ └────►│ │ │ RM │ │ RM │ │ │ │ │ └────────┘ └──────────┘ 两阶段RM协议（2PC，Two Phase Commitment Protocol），事务管理器（协调者，TM）分两个阶段来协调资源管理器（参与者，RM）： 第一阶段准备资源 是预留事务所需的资源。协调者节点向所有参与者节点询问是否可以执行提交操作。 第二阶段提交执行 分为两种情况 若参与者全部预留资源成功，则全局提交事物 若存在某些参与者资源预留失败，则全局回滚 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:1:1","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#2pc"},{"categories":["数据库"],"collections":null,"content":"TCC ┌──────────────────┐ ┌─────────────────────┐ │ │ │ │ │ TM │ ┌─┼─►try ┌────────┐ │ │ │ │ │ │ │ │ │ ┌────────┐ │ ┌───┼─┼─►confirm │service1│ │ │ │ │ │ │ │ │ │ │ │ │ │ step 1 ├─────┼─┼───┤ │ cancel └────────┘ │ │ │ │ │ ├───┼─┼─► │ │ └────────┘ │ │ │ └─────────────────────┘ ┌───────┐ │ │ │ │ │ start ├────►│ │ │ │ └───────┘ │ │ │ │ ┌─────────────────────┐ │ ┌─────────┐ │ │ │ │ │ │ │ │ │ │ └─┼─►try ┌────────┐ │ │ │ step 2 │ │ │ │ │ │ │ │ │ ├────┼─┼─────┼─►confirm │service1│ │ │ └─────────┘ │ │ │ │ │ │ │ │ └─────┼─►cancel └────────┘ │ │ │ │ │ └──────────────────┘ └─────────────────────┘ TCC（Try-Confirm-Cancel）是服务具体化的2PC约定，业务需要实现三个操作： Try 尝试预留资源 Confirm 业务提交 Cancel 释放预留资源 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:1:2","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#tcc"},{"categories":["数据库"],"collections":null,"content":"Sage normal trans compensating trans ┌────────────┐ ┌────────────┐ │ │ │ │ │ T1 │ │ C1 │ │ │ │ │ └──────┬─────┘ └─────▲──────┘ │ │ │ │ ┌──────▼─────┐ ┌─────┴──────┐ │ │ │ │ │ T2 │ │ C2 │ │ │ │ │ └──────┬─────┘ └─────▲──────┘ │ │ │ fail │ ├───────────────────────────►│ │ │ ┌──────▼─────┐ ┌─────┴──────┐ │ │ │ │ │ T3 │ │ C3 │ │ │ │ │ └────────────┘ └────────────┘ Sage是一种补偿协议，在分布式事务执行过程中，依次执行各参与者的正向操作，如果所有正向操作均执行成功，那么分布式事务提交。如果任何一个正向操作执行失败，那么分布式事务会退回去执行前面各参与者的逆向回滚操作，回滚已提交的参与者，使分布式事务回到初始状态。 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:1:3","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#sage"},{"categories":["数据库"],"collections":null,"content":"Seata框架 Seata（Simple Extensible Autonomous Transaction Architecture，简单可扩展自治事务框架），Seata中主要的三大模块为： 事务协调者（TC，Transaction Coordinator） 维护全局和分支事务的状态，驱动全局事务提交或回滚。 事务管理器（TM，Transaction Manager） 定义全局事务的范围：开始全局事务、提交或回滚全局事务。 资源管理器（RM，Resource Manager） 管理分支事务处理的资源，与TC交谈以注册分支事务和报告分支事务的状态，并驱动分支事务提交或回滚。 其中 TM、RM 作为客户端与业务集成，TC为服务端单独部署。 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:2:0","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#seata框架"},{"categories":["数据库"],"collections":null,"content":"AT模式 AT模式是一种无侵入的分布式事务解决方案。在 AT 模式下，Seata 框架会自动生成事务的二阶段提交和回滚操作。 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:2:1","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#at模式"},{"categories":["数据库"],"collections":null,"content":"TCC模式 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:2:2","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#tcc模式"},{"categories":["数据库"],"collections":null,"content":"Sage模式 ","date":"2021-08-12","objectID":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/:2:3","tags":["数据库"],"title":"分布式事务","uri":"/posts/%E5%88%86%E5%B8%83%E5%BC%8F%E4%BA%8B%E5%8A%A1/#sage模式"},{"categories":["算法"],"collections":null,"content":"链表基础算法题目 ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:0:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#"},{"categories":["算法"],"collections":null,"content":"删除排序链表中的重复元素 删除排序链表中的重复元素 给定一个排序链表，删除所有重复的元素，使得每个元素只出现一次。\\ 递归 func deleteDuplicates(head *ListNode) *ListNode { if head == nil || head.Next == nil { return head } next := head for next != nil \u0026\u0026 next.Val == head.Val { next = next.Next } head.Next = deleteDuplicates(next) // head的下个一节点是与head值不同的节点 return head } 迭代 func deleteDuplicates(head *ListNode) *ListNode { if head == nil || head.Next == nil { return head } pre, cur := head, head.Next for cur != nil { if pre.Val == cur.Val { // 若与前一个元素相同，则跳过 cur = cur.Next } else { pre.Next = cur pre = cur } } pre.Next = nil // pre之后的有可能是与pre相同的若干节点，故断开后续节点 return head } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:1:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#删除排序链表中的重复元素"},{"categories":["算法"],"collections":null,"content":"删除排序链表中的重复元素 II 删除排序链表中的重复元素 II 给定一个排序链表，删除所有含有重复数字的节点，只保留原始链表中 没有重复出现 的数字。 递归 func deleteDuplicates(head *ListNode) *ListNode { if head == nil || head.Next == nil { return head } if head.Val == head.Next.Val { for head != nil \u0026\u0026 head.Next != nil \u0026\u0026 head.Val == head.Next.Val { head = head.Next // head 与 head.Next值相同时，head指针无需保留，故向后移动 } return deleteDuplicates(head.Next) // 重复的元素不保留 } head.Next = deleteDuplicates(head.Next) return head } 迭代 func deleteDuplicates(head *ListNode) *ListNode { if head == nil || head.Next == nil { return head } dummyHead := \u0026ListNode{0, head} // 头节点可能会被删除，故引入虚拟头节点 head = dummyHead rmV := 0 // 重复值 for head.Next != nil \u0026\u0026 head.Next.Next != nil { if head.Next.Val == head.Next.Next.Val { rmV = head.Next.Val // 删除 head 后的所有与rmV相同的节点 for head.Next != nil \u0026\u0026 head.Next.Val == rmV { head.Next = head.Next.Next } } else { head = head.Next } } return dummyHead.Next } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:2:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#删除排序链表中的重复元素-ii"},{"categories":["算法"],"collections":null,"content":"反转链表 反转链表 反转一个单链表。 递归 func reverseList(head *ListNode) *ListNode { if head == nil || head.Next == nil { return head // 返回反转后的头节点 } newHead := reverseList(head.Next) // 递归获取新头节点 head.Next.Next = head // 反转当前节点和其后继节点的指向 head.Next = nil // 反转后的最后一个节点指向nil return newHead } 迭代 func reverseList(head *ListNode) *ListNode { if head == nil || head.Next == nil { return head } var pre *ListNode for head != nil { next := head.Next head.Next = pre pre = head head = next } return pre } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:3:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#反转链表"},{"categories":["算法"],"collections":null,"content":"反转链表 II 反转链表 II 反转从位置 m 到 n 的链表。请使用一趟扫描完成反转。 递归 func reverseBetween(head *ListNode, m int, n int) *ListNode { if m == 1 { return reverseN(head, n) } head.Next = reverseBetween(head.Next, m-1, n-1) // 递归移动到翻转区间 return head } var next *ListNode // 保存翻转区间后的第一个节点 // 翻转链表的前n个节点 func reverseN(head *ListNode, n int) *ListNode { if n == 1 { next = head.Next return head } last := reverseN(head.Next, n-1) head.Next.Next = head head.Next = next // 反转后的最后一个节点会指向next，其他的会被上一行覆盖 return last } 迭代 func reverseBetween(head *ListNode, m int, n int) *ListNode { if head == nil { return nil } dummyHead := \u0026ListNode{0, head} // 头节点可能被反转 // 反转区间表示为[l, r]，pre为区间外左侧第一个节点，next为右侧第一个节点 // head 移动到 l head = dummyHead var pre *ListNode // 区间左侧第一个节点 l := 0 for l \u003c m { pre = head head = head.Next l++ } lNode := head // 位于 l 的节点 var rNode *ListNode // 位于 r 的节点 （下面for循环结束后） r := l for r \u003c= n { next := head.Next head.Next = inPre rNode = head // 循环中代表head的前一个节点 head = next r++ } // 结束时 head=next 位于r+1 lNode.Next = next pre.Next = inPre return dummyHead.Next } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:4:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#反转链表-ii"},{"categories":["算法"],"collections":null,"content":"合并两个有序链表 合并两个有序链表 将两个升序链表合并为一个新的 升序 链表并返回。新链表是通过拼接给定的两个链表的所有节点组成的。 递归 func mergeTwoLists(l1 *ListNode, l2 *ListNode) *ListNode { if l1 == nil { return l2 } if l2 == nil { return l1 } var head *ListNode if l1.Val \u003c l2.Val { head = l1 head.Next = mergeTwoLists(l1.Next, l2) } else { head = l2 head.Next = mergeTwoLists(l1, l2.Next) } return head } 迭代 func mergeTwoLists(l1 *ListNode, l2 *ListNode) *ListNode { dummyHead := \u0026ListNode{0, nil} curr := dummyHead for l1 != nil \u0026\u0026 l2 != nil { if l1.Val \u003c l2.Val { curr.Next = l1 l1 = l1.Next } else { curr.Next = l2 l2 = l2.Next } curr = curr.Next } if l1 == nil \u0026\u0026 l2 == nil { return dummyHead.Next } if l1 == nil { curr.Next = l2 } if l2 == nil { curr.Next = l1 } return dummyHead.Next } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:5:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#合并两个有序链表"},{"categories":["算法"],"collections":null,"content":"分隔链表 分隔链表 给你一个链表的头节点 head 和一个特定值 x ，请你对链表进行分隔，使得所有 小于 x 的节点都出现在 大于或等于 x 的节点之前。 你应当 保留 两个分区中每个节点的初始相对位置。 func partition(head *ListNode, x int) *ListNode { if head == nil { return nil } smallHead := \u0026ListNode{0, nil} // 保存小于x的节点 small := smallHead largeHead := \u0026ListNode{0, nil} // 保存大于等于x的节点 large := largeHead for node := head; node != nil; node = node.Next { if node.Val \u003c x { small.Next = node small = small.Next } else { large.Next = node large = large.Next } } small.Next = largeHead.Next // 合并节点 large.Next = nil // 断开后续节点 return smallHead.Next } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:6:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#分隔链表"},{"categories":["算法"],"collections":null,"content":"排序链表 排序链表 O(n log n) 时间复杂度和常数级空间复杂度下对链表进行排序。 // 基于分治思想，使用归并排序 func sortList(head *ListNode) *ListNode { return sort(head, nil) } // 对区间 [left, right) 排序 func sort(left, right *ListNode) *ListNode { if left == nil { return nil } if left.Next == right { // 只有一个left节点 left.Next = nil // 断开连接所有节点的链接，在merge中会排序后再次连接 return left } // 快慢指针寻找中间节点 slow, fast := left, left for fast != right \u0026\u0026 fast.Next != right { // 右侧结尾是right节点 slow = slow.Next fast = fast.Next.Next } mid := slow return merge(sort(left, mid), sort(mid, right)) } func merge(head1, head2 *ListNode) *ListNode { dummyHead := \u0026ListNode{0, nil} cur, cur1, cur2 := dummyHead, head1, head2 for cur1 != nil \u0026\u0026 cur2 != nil { if cur1.Val \u003c cur2.Val { cur.Next = cur1 cur1 = cur1.Next } else { cur.Next = cur2 cur2 = cur2.Next } cur = cur.Next } if cur1 != nil { cur.Next = cur1 } else if cur2 != nil { cur.Next = cur2 } return dummyHead.Next } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:7:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#排序链表"},{"categories":["算法"],"collections":null,"content":"两两交换链表中的节点 两两交换链表中的节点 给定一个链表，两两交换其中相邻的节点，并返回交换后的链表。 递归 func swapPairs(head *ListNode) *ListNode { if head == nil || head.Next == nil { return head } pre, cur, next := head, head.Next, head.Next.Next cur.Next = pre pre.Next = swapPairs(next) return cur } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:8:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#两两交换链表中的节点"},{"categories":["算法"],"collections":null,"content":"环形链表 环形链表 给定一个链表，判断链表中是否有环。 func hasCycle(head *ListNode) bool { if head == nil || head.Next == nil { return false } slow, fast := head, head for fast != nil \u0026\u0026 fast.Next != nil { slow = slow.Next fast = fast.Next.Next if slow == fast { // 在环内每执行一次快慢指针距离减1 return true } } return false } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:9:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#环形链表"},{"categories":["算法"],"collections":null,"content":"环形链表 II 环形链表 II 给定一个链表，返回链表开始入环的第一个节点。 如果链表无环，则返回 null。 原始题解 a 入环点 +----------------------#------------------X 相遇点 | | | | | | +------------------+ b 入环前节点个数为a，环内节点个数为b，快指针移动距离f，慢指针移动距离s，快慢指针相遇时有： f = 2s 快指针移动距离是慢指针的两倍 f = s + nb 双指针都走过 a 步，然后在环内绕圈直到重合，重合时 fast 比 slow 多走 环的长度整数倍 由上可得： f = 2nb fast移动了2n个环长 s = nb slow移动了n个环长 又当指针ptr处于入环点时有k = a + nb，因此只需要使slow移动a个长度则可使slow指向入环点，而a长度可以由头节点移动到入环点得到，此时ptr = slow = #，即都指向入环点。 func detectCycle(head *ListNode) *ListNode { if head == nil || head.Next == nil { return nil } slow, fast := head, head for fast != nil \u0026\u0026 fast.Next != nil { slow = slow.Next fast = fast.Next.Next if slow == fast { // 相遇点 p := head // p指向头节点开始移动a个距离 for p != slow { p = p.Next slow = slow.Next } return p } } // 无环 return nil } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:10:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#环形链表-ii"},{"categories":["算法"],"collections":null,"content":"回文链表 回文链表 判断一个链表是否为回文链表。 func isPalindrome(head *ListNode) bool { if head == nil || head.Next == nil { return true } // 快慢指针找中点 slow, fast := head, head for fast != nil \u0026\u0026 fast.Next != nil { slow = slow.Next fast = fast.Next.Next } // 递归反转链表 right := reverse(slow) // 判断回文 for left := head; left != nil \u0026\u0026 right != nil; left, right = left.Next, right.Next { if left.Val != right.Val { return false } } return true } func reverse(head *ListNode) *ListNode { if head == nil || head.Next == nil { return head } last := reverse(head.Next) head.Next.Next = head head.Next = nil return last } ","date":"2021-02-19","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/:11:0","tags":["算法基础"],"title":"算法基础-链表","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-%E9%93%BE%E8%A1%A8/#回文链表"},{"categories":["算法"],"collections":null,"content":"go实现LRU和LFU ","date":"2021-01-29","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-lru%E4%B8%8Elfu/:0:0","tags":["算法基础"],"title":"算法设计-LRU与LFU","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-lru%E4%B8%8Elfu/#"},{"categories":["算法"],"collections":null,"content":"LRU type LRUCache struct { size, capacity int head, tail *KVNode cache map[int]*KVNode } type KVNode struct { prev, next *KVNode key, value int } func NewKVNode(key, value int) *KVNode { return \u0026KVNode{ key: key, value: value, } } func Constructor(capacity int) LRUCache { lru := LRUCache{ size: 0, capacity: capacity, head: NewKVNode(0, 0), tail: NewKVNode(0, 0), cache: map[int]*KVNode{}, } lru.head.next = lru.tail lru.tail.prev = lru.head return lru } func (lru *LRUCache) Get(key int) int { if node, ok := lru.cache[key]; ok { lru.MoveToHead(node) return node.value } return -1 } func (lru *LRUCache) Put(key int, value int) { if node, ok := lru.cache[key]; ok { node.value = value lru.MoveToHead(node) return } newNode := \u0026KVNode{key: key, value: value} lru.InsertToHead(newNode) lru.size++ lru.cache[key] = newNode if lru.size \u003e lru.capacity { lastNode := lru.RemoveTail() lru.size-- delete(lru.cache, lastNode.key) } } func (lru *LRUCache) MoveToHead(node *KVNode) { lru.RemoveNode(node) lru.InsertToHead(node) } func (lru *LRUCache) InsertToHead(node *KVNode) { node.prev = lru.head node.next = lru.head.next lru.head.next.prev = node lru.head.next = node } func (lru *LRUCache) RemoveTail() *KVNode { lastNode := lru.tail.prev lru.RemoveNode(lru.tail.prev) return lastNode } func (lru *LRUCache) RemoveNode(node *KVNode) { node.prev.next = node.next node.next.prev = node.prev } ","date":"2021-01-29","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-lru%E4%B8%8Elfu/:1:0","tags":["算法基础"],"title":"算法设计-LRU与LFU","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-lru%E4%B8%8Elfu/#lru"},{"categories":["算法"],"collections":null,"content":"LFU type KVNode struct { prev, next *KVNode key, value int freq int } type DoubleList struct { head, tail *KVNode size int } func NewDoubleList() *DoubleList { dl := \u0026DoubleList{ head: \u0026KVNode{ key: 0, value: 0}, tail: \u0026KVNode{ key: 0, value: 0}, size: 0, } dl.head.next = dl.tail dl.tail.prev = dl.head return dl } func (dl *DoubleList) PushFront(node *KVNode) { node.prev = dl.head node.next = dl.head.next dl.head.next.prev = node dl.head.next = node dl.size++ } func (dl *DoubleList) RemoveBack() *KVNode { lastNode := dl.tail.prev dl.Remove(lastNode) return lastNode } func (dl *DoubleList) Remove(node *KVNode) { node.prev.next = node.next node.next.prev = node.prev dl.size-- } type LFUCache struct { cache map[int]*KVNode freq map[int]*DoubleList capacity, size, minFreq int } func Constructor(capacity int) LFUCache { return LFUCache{ cache: map[int]*KVNode{}, freq: map[int]*DoubleList{}, capacity: capacity, size: 0, minFreq: 0, } } func (lfu *LFUCache) IncFreq(node *KVNode) { preFreq := node.freq lfu.freq[preFreq].Remove(node) if preFreq == lfu.minFreq \u0026\u0026 lfu.freq[preFreq].size == 0 { lfu.minFreq++ delete(lfu.freq, preFreq) } node.freq++ if lfu.freq[node.freq] == nil { lfu.freq[node.freq] = NewDoubleList() } lfu.freq[node.freq].PushFront(node) } func (lfu *LFUCache) Get(key int) int { if node, ok := lfu.cache[key]; ok { lfu.IncFreq(node) return node.value } return -1 } func (lfu *LFUCache) Put(key int, value int) { if lfu.capacity == 0 { return } if node, ok := lfu.cache[key]; ok { node.value = value lfu.IncFreq(node) return } if lfu.size == lfu.capacity { node := lfu.freq[lfu.minFreq].RemoveBack() delete(lfu.cache, node.key) lfu.size-- // No need to update minFreq because it will be set 1 below } node := \u0026KVNode{ key: key, value: value, freq: 1} lfu.cache[key] = node if lfu.freq[1] == nil { lfu.freq[1] = NewDoubleList() } lfu.freq[1].PushFront(node) lfu.minFreq = 1 lfu.size++ } ","date":"2021-01-29","objectID":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-lru%E4%B8%8Elfu/:2:0","tags":["算法基础"],"title":"算法设计-LRU与LFU","uri":"/posts/%E7%AE%97%E6%B3%95%E5%9F%BA%E7%A1%80-lru%E4%B8%8Elfu/#lfu"},{"categories":["MQTT"],"collections":null,"content":"使用openssl生成自签名证书并配置Mosquitto双向验证 ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:0:0","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#"},{"categories":["MQTT"],"collections":null,"content":"Openssl 自签CA证书 ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:1:0","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#openssl-自签ca证书"},{"categories":["MQTT"],"collections":null,"content":"生成根证书和密钥 openssl req -new -x509 -newkey rsa:2048 \\ -keyout caKey.pem -out caCrt.pem -days 365 \\ -subj \"/C=CN/ST=BJ/L=BJ/O=ZJ/OU=ROOT/CN=localhost/emailAddress=hello@world.com\" ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:1:1","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#生成根证书和密钥"},{"categories":["MQTT"],"collections":null,"content":"客户端生成密钥及证书签名请求 # mosquitto server 所需证书 openssl req -new -newkey rsa:2048 \\ -keyout hubKey.pem -out hub.csr \\ -subj \"/C=CN/ST=BJ/L=BJ/O=ZJ/OU=HUB/CN=localhost/emailAddress=hello@world.com\" # 设备所需证书 openssl req -new -newkey rsa:2048 \\ -keyout fakeMqttKey.pem -out fakeMqtt.csr \\ -subj \"/C=CN/ST=BJ/L=BJ/O=ZJ/OU=FakeMqtt/CN=localhost/emailAddress=hello@world.com\" ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:1:2","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#客户端生成密钥及证书签名请求"},{"categories":["MQTT"],"collections":null,"content":"使用 CA 根证书签发客户端证书 将两个csr文件上传至CA服务器后，由CA服务器执行签发 openssl x509 -req -days 365 \\ -in hub.csr -out hubCrt.pem \\ -CA caCrt.pem -CAkey caKey.pem \\ -CAcreateserial 同理对 fakeMqtt.csr 进行签发 openssl x509 -req -days 365 \\ -in fakeMqtt.csr -out fakeMqttCrt.pem \\ -CA caCrt.pem -CAkey caKey.pem ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:1:3","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#使用-ca-根证书签发客户端证书"},{"categories":["MQTT"],"collections":null,"content":"验证生成的证书 openssl verify -CAfile caCrt.pem hubCrt.pem openssl verify -CAfile caCrt.pem fakeMqttCrt.pem TODO：证书吊销 CRL 此时目录如下（测试时CA服务和客户端位于同一目录）： \u003e tree . ├── caCrt.pem ├── caCrt.srl ├── caKey.pem ├── fakeMqttCrt.pem ├── fakeMqttKey.pem ├── fakeMqttKeyNoPass.pem ├── hubCrt.pem └── hubKey.pem ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:1:4","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#验证生成的证书"},{"categories":["MQTT"],"collections":null,"content":"配置Mosquitto ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:2:0","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#配置mosquitto"},{"categories":["MQTT"],"collections":null,"content":"启用双向验证 原始配置文件注释比较多，因此可以新建一份配置文件用于配置 # 查看可配置的选项 cat /usr/local/etc/mosquitto/mosquitto.conf 新建 hub.conf 配置文件，其中3个file文件选项指向自己创建的密钥文件： port 1883 cafile certs/caCrt.pem certfile certs/hubCrt.pem keyfile certs/hubKey.pem require_certificate true use_identity_as_username true ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:2:1","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#启用双向验证"},{"categories":["MQTT"],"collections":null,"content":"启动 mosquitto -c ./hub.conf ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:2:2","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#启动"},{"categories":["MQTT"],"collections":null,"content":"订阅发布测试 # 订阅 mosquitto_sub -h localhost -p 1883 -t /t \\ --cafile certs/caCrt.pem \\ --cert certs/fakeMqttCrt.pem \\ --key certs/fakeMqttKey.pem # 发布 mosquitto_pub -h localhost -p 1883 -t /t -m \"hello\" \\ --cafile certs/caCrt.pem \\ --cert certs/fakeMqttCrt.pem \\ --key certs/fakeMqttKey.pem ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:2:3","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#订阅发布测试"},{"categories":["MQTT"],"collections":null,"content":"Go测试 Golang中的私钥读取无法传入密码选项，因此可以不添加密码或使用以下命令附加密码至pem文件 openssl rsa -in fakeMqttKey.pem -out fakeMqttKeyNoPass.pem -passin pass:your_password package main import ( \"crypto/tls\" \"crypto/x509\" \"fmt\" \"io/ioutil\" \"time\" mqtt \"github.com/eclipse/paho.mqtt.golang\" ) func NewTLSConfig() *tls.Config { // Import trusted certificates from CAfile.pem. // Alternatively, manually add CA certificates to // default openssl CA bundle. certPool := x509.NewCertPool() pemCerts, err := ioutil.ReadFile(\"certs/caCrt.pem\") if err == nil { certPool.AppendCertsFromPEM(pemCerts) } // Import client certificate/key pair cert, err := tls.LoadX509KeyPair( \"certs/fakeMqttCrt.pem\", \"certs/fakeMqttKeyNoPass.pem\") if err != nil { panic(err) } // Just to print out the client certificate.. cert.Leaf, err = x509.ParseCertificate(cert.Certificate[0]) if err != nil { panic(err) } fmt.Println(cert.Leaf) // Create tls.Config with desired tls properties return \u0026tls.Config{ // RootCAs = certs used to verify server cert. RootCAs: certPool, // ClientAuth = whether to request cert from server. // Since the server is set up for SSL, this happens // anyways. ClientAuth: tls.NoClientCert, // ClientCAs = certs used to validate client cert. ClientCAs: nil, // InsecureSkipVerify = verify that cert contents // match server. IP matches what is in cert etc. InsecureSkipVerify: true, // Certificates = list of certs client sends to server. Certificates: []tls.Certificate{cert}, } } var f mqtt.MessageHandler = func(client mqtt.Client, msg mqtt.Message) { fmt.Printf(\"TOPIC: %s\\n\", msg.Topic()) fmt.Printf(\"MSG: %s\\n\", msg.Payload()) } func main() { tlsconfig := NewTLSConfig() opts := mqtt.NewClientOptions() opts.AddBroker(\"ssl://localhost:1883\") opts.SetClientID(\"ssl-sample\").SetTLSConfig(tlsconfig) opts.SetDefaultPublishHandler(f) // Start the connection c := mqtt.NewClient(opts) if token := c.Connect(); token.Wait() \u0026\u0026 token.Error() != nil { panic(token.Error()) } c.Subscribe(\"/go-mqtt/sample\", 0, nil) i := 0 for range time.Tick(time.Duration(1) * time.Second) { if i == 5 { break } text := fmt.Sprintf(\"this is msg #%d!\", i) c.Publish(\"/go-mqtt/sample\", 0, false, text) i++ } c.Disconnect(250) } ","date":"2020-10-14","objectID":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/:2:4","tags":["MQTT","Mosquitto"],"title":"Mosquitto证书双向验证","uri":"/posts/mosquitto%E8%AF%81%E4%B9%A6%E5%8F%8C%E5%90%91%E9%AA%8C%E8%AF%81/#go测试"},{"categories":[],"collections":null,"content":"务必先阅读 Hbuildx官方文档 ","date":"2020-10-10","objectID":"/posts/hbuildx%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ios/:0:0","tags":[],"title":"Hbuildx本地打包iOS","uri":"/posts/hbuildx%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ios/#"},{"categories":[],"collections":null,"content":"配置SDK 打包使用的SDK版本与Hbuildx的版本一一对应，需要注意版本的匹配 按文档中的说明修改应用名称、图片等信息 HBuildx中的项目ID需要与control.xml中的id相同 ","date":"2020-10-10","objectID":"/posts/hbuildx%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ios/:1:0","tags":[],"title":"Hbuildx本地打包iOS","uri":"/posts/hbuildx%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ios/#配置sdk"},{"categories":[],"collections":null,"content":"iOS打包脚本 每次导出web资源后，需要手动在xcode中导入www并执行打包上传操作，其实这一步是可以实现自动化的，脚本如下： #!/bin/bash work_dir=`pwd` ios_sdk=\"[Your SDK Dir]\" project_dir=\"${work_dir}/${ios_sdk}/HBuilder-Hello\" xcode_project=\"${project_dir}/HBuilder-Hello.xcodeproj\" www_id=\"[Your HBuildx ID]\" www=\"${work_dir}/iwop-app/unpackage/resources/${www_id}\" archive_dir=\"${work_dir}/archive\" released_dir=\"${work_dir}/release\" export_options=\"${work_dir}/ExportOptions.plist\" workspace=\"${xcode_project}/project.xcworkspace\" plist=\"${project_dir}/HBuilder-Hello/HBuilder-Hello-Info.plist\" scheme=\"HBuilder\" mode=\"Release\" appleApiKey=\"[Your Apple Api Key]\" appleApiIssuer=\"[Your Apple User UUID]\" pgyApiKey=\"[Your Pyger ID]\" pgyUserKey=\"[Your Pyger Key]\" pgyUrl=\"https://www.pgyer.com/apiv1/app/upload\" # Colorful echo echo_green() { echo -e \"\\033[32m$1\\033[0m\" } # Verify status old_version=`/usr/libexec/Plistbuddy -c \"Print CFBundleShortVersionString\" \"${plist}\"` old_build_version=`/usr/libexec/Plistbuddy -c \"Print CFBundleVersion\" \"${plist}\"` provisioning_profiles=`/usr/libexec/Plistbuddy -c \"Print provisioningProfiles\" \"${export_options}\"` echo_green \"App current provisioning profiles: ${provisioning_profiles}\" echo_green \"App current version ${old_version}\" echo_green \"App current buildVersion ${old_build_version}\" # Update version read -p \"Input new version: \" version read -p \"Input new build verions: \" build_version if [ $version ] \u0026\u0026 [ $build_version ]; then /usr/libexec/Plistbuddy -c \"Set CFBundleShortVersionString ${version}\" \"${plist}\" /usr/libexec/Plistbuddy -c \"Set CFBundleVersion $build_version\" \"${plist}\" else version=${old_version} build_version=${old_build_version} fi echo_green \"Use the verion ${version}, build version ${build_version}\" app_version_info=\"app_v${version}_b${build_version}\" ipa_dir=\"${released_dir}/v${build_version}\" ipa=\"${ipa_dir}/HBuilder.ipa\" upadte_www_resource() { echo_green \"Delete the old ${project_dir}/HBuilder-Hello/Pandora/apps/${www_id}\" rm -r \"${project_dir}/HBuilder-Hello/Pandora/apps/${www_id}\" echo_green \"Add new ${www} to ${project_dir}/HBuilder-Hello/Pandora/apps/\" cp -r ${www} \"${project_dir}/HBuilder-Hello/Pandora/apps/\" } clean_xcode_build_cache() { echo_green \"Clean the build cache\" xcodebuild clean -workspace ${workspace} -scheme ${scheme} -configuration ${mode} } build_ios_installer_ipa() { upadte_www_resource clean_xcode_build_cache echo_green \"Building app...\" archive_file=\"${archive_dir}/${scheme}v${build_version}.xcarchive\" xcodebuild -quiet archive -workspace ${workspace} -scheme ${scheme} -archivePath ${archive_file} xcodebuild -quiet -exportArchive -archivePath ${archive_file} -exportPath \"${released_dir}/v${build_version}\" -exportOptionsPlist \"${export_options}\" } open_ipa_folder() { read -p \"Open ipa folder? Enter to continue\" open ${ipa_dir} } upload_pyger() { read -p \"Upload to he pgyer.com? Enter to continue\" echo_green \"Uploading ${ipa}\" curl -F \"file=@${ipa}\" -F \"uKey=${pgyUserKey}\" -F \"_api_key=${pgyApiKey}\" ${pgyUrl} echo \"\" echo_green \"https://www.pgyer.com/dvuF\" } verify_ipa() { echo_green \"Verifying the ipa...\" xcrun altool --validate-app -f ${ipa} -t \"iOS\" --apiKey ${appleApiKey} --apiIssuer ${appleApiIssuer} } upload_apple_store() { read -p \"Upload to apple store? Enter to continue\" xcrun altool --upload-app -f \"${released_dir}/v${build_version}/HBuilder.ipa\" -t \"iOS\" --apiKey ${appleApiKey} --apiIssuer ${appleApiIssuer} } open_xcode() { read -p \"Open the Xcode? Enter to continue\" open ${xcode_project} } showTips() { echo -e \"\\033[32mPlease change push cert to development mode: manifest.json =\u003e app module =\u003e push =\u003e setting\\033[0m\" } while : do echo_green \"Choose the action (${app_version_info}) uw) upadte_www_resource;; cx) clean_xcode_build_cache;; bi) build_ios_installer_ipa;; vi) verify_ipa;; oi) open_ipa_folder;; up) upload_pyger;; ua) upload_apple_store;; ox) open_xcode;; q) quit;;\" read -p \"\" op case $op in uw) upadte_www_resource;; cx) clean_xcode_build_cache;; bi) build_ios_installer_ipa;; vi) verify_","date":"2020-10-10","objectID":"/posts/hbuildx%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ios/:2:0","tags":[],"title":"Hbuildx本地打包iOS","uri":"/posts/hbuildx%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ios/#ios打包脚本"},{"categories":[],"collections":null,"content":"企业分发 若开发者账户类型为企业开发者,则可以通过In House模式打包可以直接在Safari中发布应用.ExportOptions.plist参考如下: \u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e \u003c!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\"\u003e \u003cplist version=\"1.0\"\u003e \u003cdict\u003e \u003ckey\u003ecompileBitcode\u003c/key\u003e \u003cfalse/\u003e \u003ckey\u003edestination\u003c/key\u003e \u003cstring\u003eexport\u003c/string\u003e \u003ckey\u003emanifest\u003c/key\u003e \u003cdict\u003e \u003ckey\u003eappURL\u003c/key\u003e \u003cstring\u003e[Ipa文件Url]\u003cstring\u003e \u003ckey\u003edisplayImageURL\u003c/key\u003e \u003cstring\u003e[57x57分辨率logo的Url]\u003c/string\u003e \u003ckey\u003efullSizeImageURL\u003c/key\u003e \u003cstring\u003e[512x512分辨率logo的Url]\u003c/string\u003e \u003c/dict\u003e \u003ckey\u003emethod\u003c/key\u003e \u003cstring\u003eenterprise\u003c/string\u003e \u003ckey\u003eprovisioningProfiles\u003c/key\u003e \u003cdict\u003e \u003ckey\u003e[Bundle ID]\u003c/key\u003e \u003cstring\u003e[Name]\u003c/string\u003e \u003c/dict\u003e \u003ckey\u003esigningCertificate\u003c/key\u003e \u003cstring\u003eApple Distribution\u003c/string\u003e \u003ckey\u003esigningStyle\u003c/key\u003e \u003cstring\u003emanual\u003c/string\u003e \u003ckey\u003estripSwiftSymbols\u003c/key\u003e \u003ctrue/\u003e \u003ckey\u003eteamID\u003c/key\u003e \u003cstring\u003e[Team ID]\u003c/string\u003e \u003ckey\u003ethinning\u003c/key\u003e \u003cstring\u003e\u0026lt;none\u0026gt;\u003c/string\u003e \u003c/dict\u003e \u003c/plist\u003e 其中三个Url地址需要是Https开头,否则无法正常安装. 随后随便写一个发布页面挂在网站上即可,用户通过Safari访问后即可安装对应应用, Html页面参考: \u003c!DOCTYPE html\u003e \u003chtml\u003e \u003chead\u003e \u003cmeta charset=\"utf-8\" /\u003e \u003ctitle\u003e应用名称\u003c/title\u003e \u003clink href=\"https://cdn.jsdelivr.net/npm/bootstrap@5.0.0-beta1/dist/css/bootstrap.min.css\" rel=\"stylesheet\" integrity=\"sha384-giJF6kkoqNQ00vy+HMDP7azOuL0xtbfIcaT9wjKHr8RbDVddVHyTfAAsrekwKmP1\" crossorigin=\"anonymous\"\u003e \u003cscript src=\"https://cdn.jsdelivr.net/npm/bootstrap@5.0.0-beta1/dist/js/bootstrap.bundle.min.js\" integrity=\"sha384-ygbV9kiqUc6oa4msXn9868pTtWMgiQaeYH7/t7LECLbyPA2x65Kgf80OJFdroafW\" crossorigin=\"anonymous\"\u003e\u003c/script\u003e \u003c/head\u003e \u003cbody\u003e \u003cdiv class=\"container\" style=\"width: 50rem;\"\u003e \u003cdiv class=\"row \"\u003e \u003cdiv class=\"col\"\u003e \u003cp class=\"text-center fs-4 fw-bold\"\u003e应用名称\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"row\"\u003e \u003cdiv class=\"col\"\u003e \u003cdiv class=\"card mx-auto\" style=\"width: 20rem;\"\u003e \u003cimg src=\"./static/apple.svg\" class=\"card-img-top mx-auto\" style=\"width: 12rem;\"\u003e \u003cdiv class=\"card-body text-center\"\u003e \u003cp class=\"card-text mx-auto\"\u003ePlease open this page via safari.\u003c/p\u003e \u003ca href=\"itms-services://?action=download-manifest\u0026url=[manifest.plist的Url]\" class=\"btn btn-primary\"\u003eDownload iOS\u003c/a\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"col\"\u003e \u003cdiv class=\"card mx-auto\" style=\"width: 20rem;\"\u003e \u003cimg src=\"./static/android.svg\" class=\"card-img-top mx-auto\" style=\"width: 12rem;\"\u003e \u003cdiv class=\"card-body text-center\"\u003e \u003cp class=\"card-text\"\u003ePlease open this page via any browser.\u003c/p\u003e \u003ca href=\"[Android Apk 的 Url]\" class=\"btn btn-primary\"\u003eDownload Android\u003c/a\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e \u003c/body\u003e \u003c/html\u003e 最重要的是 \u003ca href=“itms-services://?action=download-manifest\u0026url=[manifest.plist的Url] 这一行 ","date":"2020-10-10","objectID":"/posts/hbuildx%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ios/:3:0","tags":[],"title":"Hbuildx本地打包iOS","uri":"/posts/hbuildx%E6%9C%AC%E5%9C%B0%E6%89%93%E5%8C%85ios/#企业分发"},{"categories":["时序数据库"],"collections":null,"content":"基于原生安装方式进行部署 部署进程模式下的ClickHouse ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:0:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#"},{"categories":["时序数据库"],"collections":null,"content":"服务器 s1r1（分片1副本1）、s3r2（分片3副本2） 位于 s1 服务器 s2r1（分片2副本1）、s1r2 （分片1副本2）位于 s2 服务器 s3r1（分片3副本1）、s2r2（分片2副本2） 位于 s3 服务器 zoo1（位于s1）、zoo2（位于s2）、zoo3 （位于s3）为 zookeeper集群 注意端口限制，有些可能需要手动设置防火墙放行端口 ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:1:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#服务器"},{"categories":["时序数据库"],"collections":null,"content":"设置Host 在 s1、s2、s3中的/etc/hosts 中添加以下内容（需替换为自己的IP） s1-ip s1 s2-ip s2 s3-ip s3 ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:2:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#设置host"},{"categories":["时序数据库"],"collections":null,"content":"配置Zookeeper ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:3:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#配置zookeeper"},{"categories":["时序数据库"],"collections":null,"content":"下载 Zookeeper官网 # 使用国内镜像源，下载前请确认当前版本在镜像中存在 wget https://mirrors.bfsu.edu.cn/apache/zookeeper/zookeeper-3.6.2/apache-zookeeper-3.6.2-bin.tar.gz tar -zxf apache-zookeeper-3.6.2-bin.tar.gz \u0026\u0026 mv apache-zookeeper-3.6.2-bin zookeeper cd zookeeper \u0026\u0026 cp conf/zoo_sample.cfg conf/zoo.cfg \u0026\u0026 vim conf/zoo.cfg ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:3:1","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#下载"},{"categories":["时序数据库"],"collections":null,"content":"配置 修改 conf/zoo.cfg 内容为： # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/data/zookeeper # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to \"0\" to disable auto purge feature #autopurge.purgeInterval=1 ## Metrics Providers # # https://prometheus.io Metrics Exporter #metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider #metricsProvider.httpPort=7000 #metricsProvider.exportJvmInfo=true ## Cluster 如果是云服务器需要使用内网地址ip或在白名单中的hostname server.1=s1:2888:3888 server.2=s2:2888:3888 server.3=s3:2888:3888 修改conf/log4j.properties zookeeper.log.dir=/data/log/zookeeper 在～/.bashrc中添加 export ZOO_LOG_DIR=/data/log/zookeeper 在dataDir对应目录下新建myid文件，分别在s1、s2、s3上填入 1 2 3 ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:3:2","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#配置"},{"categories":["时序数据库"],"collections":null,"content":"启动 bin/zkServer.sh start 执行命令验证安装结果： bin/zkServer.sh status ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:3:3","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#启动"},{"categories":["时序数据库"],"collections":null,"content":"安装ClickHouse 官方文档 sudo apt-get install apt-transport-https ca-certificates dirmngr sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E0C56BD4 echo \"deb https://repo.clickhouse.tech/deb/stable/ main/\" | sudo tee \\ /etc/apt/sources.list.d/clickhouse.list sudo apt-get update sudo apt-get install -y clickhouse-server clickhouse-client sudo service clickhouse-server start clickhouse-client 手动安装 wget https://mirrors.tuna.tsinghua.edu.cn/clickhouse/deb/stable/main/clickhouse-common-static_20.9.7.11_amd64.deb wget https://mirrors.tuna.tsinghua.edu.cn/clickhouse/deb/stable/main/clickhouse-client_20.9.7.11_all.deb wget https://mirrors.tuna.tsinghua.edu.cn/clickhouse/deb/stable/main/clickhouse-server_20.9.7.11_all.deb dpkg -i clickhouse-common-static_20.9.7.11_amd64.deb dpkg -i clickhouse-client_20.9.7.11_all.deb dpkg -i clickhouse-server_20.9.7.11_all.deb ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:4:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#安装clickhouse"},{"categories":["时序数据库"],"collections":null,"content":"ClickHouse工作目录 # s1 ├── data_p.csv #（测试用）示例数据 ├── s1r1 │ ├── config.xml # 配置文件 │ ├── metrika.xml # 分片配置文件 ├── s3r2 │ ├── config.xml │ ├── metrika.xml └── users.xml # s2 ├── s1r2 │ ├── config.xml │ ├── metrika.xml ├── s2r1 │ ├── config.xml │ ├── metrika.xml └── users.xml # s3 ├── s2r2 │ ├── config.xml │ ├── metrika.xml ├── s3r1 │ ├── config.xml │ ├── metrika.xml └── users.xml 配置中tcp访问端口如下： s1r1:9111 s3r2:9132 s2r1:9121 s1r2:9112 s3r1:9131 s2r2:9122 配置中http访问端口（datagrip工具使用此端口通信）如下： s1r1:9011 s3r2:9032 s2r1:9021 s1r2:9012 s3r1:9031 s2r2:9022 ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:5:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#clickhouse工作目录"},{"categories":["时序数据库"],"collections":null,"content":"运行ClickHouse 停止默认clickhouse sudo service clickhouse stop 运行clickhouse # s1 nohup clickhouse-server --config-file=/data/ch/s1r1/config.xml \u003e /data/log/ch/s1r1.log 2\u003e\u00261 \u0026 nohup clickhouse-server --config-file=/data/ch/s3r2/config.xml \u003e /data/log/ch/s3r2.log 2\u003e\u00261 \u0026 # s2 nohup clickhouse-server --config-file=/data/ch/s1r2/config.xml \u003e /data/log/ch/s1r2.log 2\u003e\u00261 \u0026 nohup clickhouse-server --config-file=/data/ch/s2r1/config.xml \u003e /data/log/ch/s2r1.log 2\u003e\u00261 \u0026 # s3 nohup clickhouse-server --config-file=/data/ch/s2r2/config.xml \u003e /data/log/ch/s2r2.log 2\u003e\u00261 \u0026 nohup clickhouse-server --config-file=/data/ch/s3r1/config.xml \u003e /data/log/ch/s3r1.log 2\u003e\u00261 \u0026 停止运行： ps -A | grep clickhouse | grep -v grep | awk ‘{print $1}’ | xargs kill ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:6:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#运行clickhouse"},{"categories":["时序数据库"],"collections":null,"content":"自定义用户名和密码 ** 在metrika.xml中需要设置密码，注意同步 ** 在users.xml中有详细的注释说明了如何添加用户、密码、用户权限等配置方法。（配置文件中已添加了以下内容，需要其他账户可参考以下步骤） 添加defualt用户密码和新建一个tom用户，由于密码存储在文件，因此推荐使用sha256编码后放入文件： echo -n \"default\" | sha256sum | tr -d '-' # 37a8eec1ce19687d132fe29051dca629d164e2c4958ba141d5f4133a33f0688f echo -n \"tom-password\" | sha256sum | tr -d '-' # d8c862690b30f6f2add244327715cb08ac926c7c2fb4fcbb7694650bfde5b672 default用户密码为default，tom的密码为tom-password，添加至users.xml，在 users/default中删除password项，添加： \u003cpassword_sha256_hex\u003e37a8eec1ce19687d132fe29051dca629d164e2c4958ba141d5f4133a33f0688f\u003c/password_sha256_hex\u003e 在users/下添加： \u003ctom\u003e \u003cpassword_sha256_hex\u003ed8c862690b30f6f2add244327715cb08ac926c7c2fb4fcbb7694650bfde5b672\u003c/password_sha256_hex\u003e \u003cprofile\u003edefault\u003c/profile\u003e \u003cquota\u003edefault\u003c/quota\u003e \u003cnetworks incl=\"networks\" replace=\"replace\"\u003e \u003cip\u003e::/0\u003c/ip\u003e \u003c/networks\u003e \u003c/tom\u003e 登陆数据库： clickhouse-client --host s1 -u default --password default --port 9111 clickhouse-client --host s1 --user tom --password tom-password --port 9111 ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:7:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#自定义用户名和密码"},{"categories":["时序数据库"],"collections":null,"content":"测试 登陆任意副本数据库，创建测试表 -- DROP TABLE p_local ON CLUSTER p_3shards_2replicas CREATE TABLE p_local ON CLUSTER p_3shards_2replicas ( ozone Int64, particullate_matter Int8, carbon_monoxide Int8, sulfure_dioxide Int8, nitrogen_dioxide Int8, longitude Float64, latitude Float64, timestamp DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/default/tables/p/{shard}','{replica}') PARTITION BY toYYYYMM(timestamp) ORDER BY timestamp PRIMARY KEY timestamp 为表p_local创建分布表p -- DROP TABLE p ON CLUSTER p_3shards_2replicas CREATE TABLE p ON CLUSTER p_3shards_2replicas AS p_local ENGINE = Distributed(p_3shards_2replicas, default, p_local, rand()) 退出数据库交互界面，在终端中执行语句导入数据 clickhouse-client --host s1 -u default --password default --port 9111 --query \"INSERT INTO p FORMAT CSV\" \u003c data_p.csv 删除测试表 drop table p on cluster p_3shards_2replicas drop table p_local on cluster p_3shards_2replicas ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:8:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#测试"},{"categories":["时序数据库"],"collections":null,"content":"配置kafka ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:9:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#配置kafka"},{"categories":["时序数据库"],"collections":null,"content":"下载 Kafka官网 # 使用国内镜像源，下载前请确认当前版本在镜像中存在 wget https://mirrors.bfsu.edu.cn/apache/kafka/2.6.0/kafka_2.13-2.6.0.tgz tar -zxf kafka_2.13-2.6.0.tgz \u0026\u0026 mv kafka_2.13-2.6.0 kafka ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:9:1","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#下载-1"},{"categories":["时序数据库"],"collections":null,"content":"配置 修改配置 cd kafka \u0026\u0026 vim config/server.properties 通用配置 # A comma separated list of directories under which to store log files log.dirs=/data/log/kafka # root directory for all kafka znodes. zookeeper.connect=s1:2181,s2:2181,s3:2181 分节点配置 s1 broker.id=1 listeners=PLAINTEXT://s1:9092 s2 broker.id=2 listeners=PLAINTEXT://s2:9092 s3 broker.id=3 listeners=PLAINTEXT://s3:9092 ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:9:2","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#配置-1"},{"categories":["时序数据库"],"collections":null,"content":"启动 ./bin/kafka-server-start.sh -daemon ./config/server.properties 查看topic bin/kafka-topics.sh --list --bootstrap-server s1:9092 ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:9:3","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#启动-1"},{"categories":["时序数据库"],"collections":null,"content":"添加kafka缓冲 创建kafka主题 # ./bin/kafka-topics.sh --delete --bootstrap-server s1:9092 --topic topic-p ./bin/kafka-topics.sh --create --bootstrap-server s1:9092 --replication-factor 2 --partitions 6 --topic topic-p 创建kafka消费表 -- DROP TABLE p_stream ON CLUSTER p_3shards_2replicas CREATE TABLE p_stream ON CLUSTER p_3shards_2replicas ( ozone Int64, particullate_matter Int8, carbon_monoxide Int8, sulfure_dioxide Int8, nitrogen_dioxide Int8, longitude Float64, latitude Float64, timestamp DateTime ) ENGINE = Kafka() SETTINGS kafka_broker_list = 's1:9092', kafka_topic_list = 'topic-p', kafka_group_name = 'clickhouse', kafka_format = 'JSONEachRow', kafka_row_delimiter = '\\n' 添加 kafka_skip_broken_messages = N 跳过非法消息 创建本地表 -- DROP TABLE p_local ON CLUSTER p_3shards_2replicas CREATE TABLE p_local ON CLUSTER p_3shards_2replicas ( ozone Int64, particullate_matter Int8, carbon_monoxide Int8, sulfure_dioxide Int8, nitrogen_dioxide Int8, longitude Float64, latitude Float64, timestamp DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/p/{shard}','{replica}') PARTITION BY toYYYYMM(timestamp) ORDER BY timestamp PRIMARY KEY timestamp 创建分布表 -- DROP TABLE p ON CLUSTER p_3shards_2replicas -- CREATE TABLE p ON CLUSTER p_3shards_2replicas AS p_local ENGINE = Distributed(p_3shards_2replicas, default, p_local, rand()) CREATE TABLE p ON CLUSTER p_3shards_2replicas ( ozone Int64, particullate_matter Int8, carbon_monoxide Int8, sulfure_dioxide Int8, nitrogen_dioxide Int8, longitude Float64, latitude Float64, timestamp DateTime ) ENGINE = Distributed(p_3shards_2replicas, default, p_local, rand()) 创建物化视图 -- DROP TABLE p_consumer ON CLUSTER p_3shards_2replicas CREATE MATERIALIZED VIEW p_consumer ON CLUSTER p_3shards_2replicas TO p AS SELECT * FROM p_stream; 发送数据 ./bin/kafka-console-producer.sh --broker-list s1:9092 --topic topic-p ","date":"2020-09-04","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/:10:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份进程模式","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E8%BF%9B%E7%A8%8B%E6%A8%A1%E5%BC%8F/#添加kafka缓冲"},{"categories":["边缘计算"],"collections":null,"content":"Baetyl v2.0.0安装指南 ","date":"2020-08-31","objectID":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/:0:0","tags":["Baetyl"],"title":"Baetyl V2安装指南","uri":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/#"},{"categories":["边缘计算"],"collections":null,"content":"Docker /etc/docker/daemon.json中添加/修改： 重启Docker ","date":"2020-08-31","objectID":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/:1:0","tags":["Baetyl"],"title":"Baetyl V2安装指南","uri":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/#docker"},{"categories":["边缘计算"],"collections":null,"content":"Kubernets ","date":"2020-08-31","objectID":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/:2:0","tags":["Baetyl"],"title":"Baetyl V2安装指南","uri":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/#kubernets"},{"categories":["边缘计算"],"collections":null,"content":"Rancher 参考官方文档和国内部署 docker run -itd -p 9080:80 -p 9443:443 \\ --name rancher \\ --restart=unless-stopped \\ -e CATTLE_AGENT_IMAGE=\"registry.cn-hangzhou.aliyuncs.com/rancher/rancher-agent:v2.4.2\" \\ registry.cn-hangzhou.aliyuncs.com/rancher/rancher:v2.4.2 清空数据 docker stop $(docker ps -aq) docker system prune -f docker volume rm $(docker volume ls -q) docker image rm $(docker image ls -q) rm -rf /etc/ceph \\ /etc/cni \\ /etc/kubernetes \\ /opt/cni \\ /opt/rke \\ /run/secrets/kubernetes.io \\ /run/calico \\ /run/flannel \\ /var/lib/calico \\ /var/lib/etcd \\ /var/lib/cni \\ /var/lib/kubelet \\ /var/lib/rancher/rke/log \\ /var/log/containers \\ /var/log/pods \\ /var/run/calico ","date":"2020-08-31","objectID":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/:2:1","tags":["Baetyl"],"title":"Baetyl V2安装指南","uri":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/#rancher"},{"categories":["边缘计算"],"collections":null,"content":"k3s 使用国内源安装 curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -s - server --docker 配置config cp /etc/rancher/k3s/k3s.yaml ~/.kube/config 添加本地存储支持： wget https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml kubectl create -f local-path-storage.yaml 设置该存储为默认存储： kubectl patch storageclass local-path -p '{\"metadata\": {\"annotations\":{\"storageclass.kubernetes.io/is-default-class\":\"true\"}}}' ","date":"2020-08-31","objectID":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/:2:2","tags":["Baetyl"],"title":"Baetyl V2安装指南","uri":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/#k3s"},{"categories":["边缘计算"],"collections":null,"content":"K8S # 安装 kubeadm kubectl kubelet gpg --keyserver keyserver.ubuntu.com --recv-keys BA07F4FB gpg --export --armor BA07F4FB | sudo apt-key add - echo \"deb https://mirrors.tuna.tsinghua.edu.cn/kubernetes/apt kubernetes-xenial main\" \u003e /etc/apt/sources.list.d/kubernetes.list apt update apt install kubeadm kubectl kubelet # 查看指定k8s版本需要哪些镜像 kubeadm config images list --kubernetes-version v1.18.3 终端输出： k8s.gcr.io/kube-apiserver:1.18.3 k8s.gcr.io/kube-controller-manager:v1.18.3 k8s.gcr.io/kube-scheduler:v1.18.3 k8s.gcr.io/kube-proxy:v1.18.3 k8s.gcr.io/pause:3.2 k8s.gcr.io/etcd:3.4.3-0 k8s.gcr.io/coredns:1.6.7 新建脚本get-k8s-images.sh 并替换版本号： #!/bin/bash images=( kube-apiserver:v1.18.3 kube-controller-manager:v1.18.3 kube-scheduler:v1.18.3 kube-proxy:v1.18.3 pause:3.2 etcd:3.4.3-0 coredns:1.6.7 ) for imageName in ${images[@]} ; do docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName done 执行get-k8s-images.sh 以便从国内hub获取镜像。修改kubelet配置中的默认cgroup driver： cat \u003e /var/lib/kubelet/config.yaml \u003c\u003cEOF apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cgroupDriver: systemd EOF systemctl restart kubelet 启动k8s： kubeadm init --pod-network-cidr=10.244.0.0/16 --kubernetes-version=v1.18.3 启动完毕后有后续步骤的相关提示，具体操作为配置$HOME/.kube/config： mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config 添加网络组件（Flannel）： wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml kubectl apply -f kube-flannel.yml 添加本地存储支持： wget https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml kubectl create -f local-path-storage.yaml 设置该存储为默认存储： kubectl patch storageclass local-path -p '{\"metadata\": {\"annotations\":{\"storageclass.kubernetes.io/is-default-class\":\"true\"}}}' ","date":"2020-08-31","objectID":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/:2:3","tags":["Baetyl"],"title":"Baetyl V2安装指南","uri":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/#k8s"},{"categories":["边缘计算"],"collections":null,"content":"安装Baetyl 参考官方文档 在安装边缘节点时报错： curl -d \"{\\\"name\\\":\\\"demo-node\\\"}\" -H \"Content-Type: application/json\" -X POST http://0.0.0.0:30004/v1/nodes {\"code\":\"UnknownError\",\"message\":\"nodes.cloud.baetyl.io \\\"demo-node\\\" is forbidden: User \\\"system:serviceaccount:default:baetyl-cloud\\\" cannot get resource \\\"nodes\\\" in API group \\\"cloud.baetyl.io\\\" in the namespace \\\"baetyl-cloud\\\"\",\"requestId\":\"\"} 临时的解决办法：为账户baetyl-cloud添加所有相关权限： kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: my-baetyl-cr labels: custom: role-patch rules: - apiGroups: - cloud.baetyl.io resources: - nodes - applications - configurations - nodedesires - nodereports - secrets verbs: - get - list - watch - create - update - patch - delete apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: my-baetyl-crb labels: custom: role-patch roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: my-baetyl-cr subjects: - kind: ServiceAccount name: baetyl-cloud namespace: default ","date":"2020-08-31","objectID":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/:3:0","tags":["Baetyl"],"title":"Baetyl V2安装指南","uri":"/posts/baetyl-v2%E5%AE%89%E8%A3%85%E6%8C%87%E5%8D%97/#安装baetyl"},{"categories":["边缘计算"],"collections":null,"content":"国内网络环境下快速安装k8s ","date":"2020-08-31","objectID":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/:0:0","tags":["Kubeedge"],"title":"Kubeedge 国内环境安装","uri":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/#"},{"categories":["边缘计算"],"collections":null,"content":"安装k8s或k3s 参考官方网站的文档即可，k8s的配置部分参考kubeedge官方文档 ","date":"2020-08-31","objectID":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/:1:0","tags":["Kubeedge"],"title":"Kubeedge 国内环境安装","uri":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/#安装k8s或k3s"},{"categories":["边缘计算"],"collections":null,"content":"配置安装环境 安装 kubeadm、kubectl： gpg --keyserver keyserver.ubuntu.com --recv-keys BA07F4FB gpg --export --armor BA07F4FB | sudo apt-key add - echo \"deb https://mirrors.tuna.tsinghua.edu.cn/kubernetes/apt kubernetes-xenial main\" \u003e /etc/apt/sources.list.d/kubernetes.list apt update apt install kubeadm kubectl ipvsadm 下载Golang安装包并解压到 /usr/local，添加环境变量： # Golang export GOROOT=/usr/local/go export GOPATH=/data/gopath export PATH=$PATH:$GOROOT/bin:$GOPATH/bin # Kubeedge export PATH=$PATH:/data/gopath/src/github.com/kubeedge/kubeedge/_output/local/bin source 环境变量文件使之生效。 ","date":"2020-08-31","objectID":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/:2:0","tags":["Kubeedge"],"title":"Kubeedge 国内环境安装","uri":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/#配置安装环境"},{"categories":["边缘计算"],"collections":null,"content":"编译Kubeedge 克隆源码（gitclone.com 用于国内加速）： git clone https://gitclone.com/github.com/kubeedge/kubeedge $GOPATH/src/github.com/kubeedge/kubeedge 由于kubeedge安编译中需要使用git的历史信息，因此将remote改回github地址： git remote set-url origin https://github.com/kubeedge/kubeedge git remote show origin 创建工作目录： mkdir -p /data/gopath \u0026\u0026 cd /data/gopath mkdir -p src pkg bin 编译： cd $GOPATH/src/github.com/kubeedge/kubeedge make all WHAT=keadm make all WHAT=cloudcore make all WHAT=edgecore 成功编译后的文件位于 ./_output/local/bin，该目录在之前的环境变量中已添加。 ","date":"2020-08-31","objectID":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/:3:0","tags":["Kubeedge"],"title":"Kubeedge 国内环境安装","uri":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/#编译kubeedge"},{"categories":["边缘计算"],"collections":null,"content":"创建cloud节点 使用keadm进行快速部署，由于安装过程需要访问github相关资源，因此选择手动提前下载所需文件： g.ioiox.com 为github资源镜像加速，若失效则需自己替换可用镜像 版本号根据当前kubeedge版本自行替换 mkdir /etc/kubeedge cd /etc/kubeedge \u0026\u0026 wget https://g.ioiox.com/https://github.com/kubeedge/kubeedge/releases/download/v1.3.1/kubeedge-v1.3.1-linux-amd64.tar.gz 同时可能需要添加 raw.githubusercontent.com 的DNS解析： 151.101.108.133 raw.githubusercontent.com 创建cloud节点： IP替换为本机IP keadm init --advertise-address=\"IP\" 查看日志： tail -f /var/log/kubeedge/cloudcore.log ","date":"2020-08-31","objectID":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/:4:0","tags":["Kubeedge"],"title":"Kubeedge 国内环境安装","uri":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/#创建cloud节点"},{"categories":["边缘计算"],"collections":null,"content":"创建edge节点 在cloud端获取token： keadm gettoken 将cloud端的./_output/local/bin二进制文件拷贝至edge端并加入PATH，随后创建edge节点： CLOUD_IP 为前文中的cloud端暴露的IP TOKEN为获取到的token keadm join --cloudcore-ipport=CLOUD_IP:10000 --token=TOKEN 查看日志： tail -f /var/log/kubeedge/edgecore.log ","date":"2020-08-31","objectID":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/:5:0","tags":["Kubeedge"],"title":"Kubeedge 国内环境安装","uri":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/#创建edge节点"},{"categories":["边缘计算"],"collections":null,"content":"删除/重置Kubeedge 停止当前运行的kubeedge组件： keadm reset 在cloud端： rm -r /etc/kubeedge kubectl delete CustomResourceDefinition $(k get CustomResourceDefinition | grep kubeedge | awk '{print $1}') 删除 /etc/kubeedge 目录后需要再次手动下载 kubeedge-v1.3.1-linux-amd64.tar.gz ","date":"2020-08-31","objectID":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/:6:0","tags":["Kubeedge"],"title":"Kubeedge 国内环境安装","uri":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/#删除重置kubeedge"},{"categories":["边缘计算"],"collections":null,"content":"验证 在cloud端： root@mq-228 /e/kubeedge# kubectl get no NAME STATUS ROLES AGE VERSION mq-228 Ready master 11d v1.18.6+k3s1 mq-227 Ready agent,edge 38m v1.17.1-kubeedge-v1.3.1 Counter Demo 未更新，按教程无法直接运行: apiVersion需要更新到v1alpha2 code.jquery.com无法在国内访问 ","date":"2020-08-31","objectID":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/:7:0","tags":["Kubeedge"],"title":"Kubeedge 国内环境安装","uri":"/posts/kubeedge-%E5%9B%BD%E5%86%85%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85/#验证"},{"categories":["时序数据库"],"collections":null,"content":"基于Docker的部署方式 ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:0:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#"},{"categories":["时序数据库"],"collections":null,"content":"安装 安装 docker 安装 docker-compose docker pull yandex/clickhouse-server docker pull zookeeper ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:1:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#安装"},{"categories":["时序数据库"],"collections":null,"content":"服务器设置 s1r1（分片1副本1）、s3r2（分片3副本2） 位于 s1 服务器 s2r1（分片2副本1）、s1r2 （分片1副本2）位于 s2 服务器 s3r1（分片3副本1）、s2r2（分片2副本2） 位于 s3 服务器 zoo1（位于s1）、zoo2（位于s2）、zoo3 （位于s3）为 zookeeper集群 /etc/hosts 映射信息，实际部署时需要替换为自己的服务器IP，本文只在s1服务器中添加hosts信息，后文中若未指明服务器名，则默认在s1中执行： [S1] s1 [S2] s2 [S3] s3 [S1] s1r1 [S2] s1r2 [S2] s2r1 [S3] s2r2 [S3] s3r1 [S1] s3r2 [S1] zoo1 [S2] zoo2 [S3] zoo3 ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:2:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#服务器设置"},{"categories":["时序数据库"],"collections":null,"content":"zookeeper 设置 使用 docker-compose 进行部署，docker-compose.yaml 文件内容如下： zoo1（s1服务器）: version: '3.1' services: zoo1: image: zookeeper restart: always hostname: zoo1 ports: - 2181:2181 - 2888:2888 - 3888:3888 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=0.0.0.0:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181 extra_hosts: - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"zoo3:[S3]\" zoo2（s2服务器）： version: '3.1' services: zoo2: image: zookeeper restart: always hostname: zoo2 ports: - 2181:2181 - 2888:2888 - 3888:3888 environment: ZOO_MY_ID: 2 ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=0.0.0.0:2888:3888;2181 server.3=zoo3:2888:3888;2181 extra_hosts: - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"zoo3:[S3]\" zoo3（s3服务器）: version: '3.1' services: zoo3: image: zookeeper restart: always hostname: zoo3 ports: - 2181:2181 - 2888:2888 - 3888:3888 environment: ZOO_MY_ID: 3 ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=0.0.0.0:2888:3888;2181 extra_hosts: - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"z003:[S3]\" 在s1、s2、s3上分别启动zookeeper： docker-compose up -d 停止容器 docker-compose stop 删除容器 docker-compose rm 显示容器 docker-compose ps 重启容器 docker-compose restart ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:3:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#zookeeper-设置"},{"categories":["时序数据库"],"collections":null,"content":"配置数据库 ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:4:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#配置数据库"},{"categories":["时序数据库"],"collections":null,"content":"导出配置文件模版 首先启动一个临时容器来获取配置文件config.xml： docker run -itd --rm --name db yandex/clickhouse-server 拷贝config.xml和users.xml： docker cp db:/etc/clickhouse-server/config.xml ./config.xml docker cp db:/etc/clickhouse-server/users.xml ./users.xml 停止容器： docker stop db config.xml在三台服务器上均需要2份（2备份），users.xml在三台服务器中各需要一份（2备份共享该配置） ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:4:1","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#导出配置文件模版"},{"categories":["时序数据库"],"collections":null,"content":"添加用户和密码 在users.xml中有详细的注释说明了如何添加用户、密码、用户权限等配置方法。这里添加defualt用户密码和新建一个tom用户，由于密码存储在文件，因此推荐使用sha256编码后放入文件： root@mq-228 ~/i/chdb# echo -n \"default\" | sha256sum | tr -d '-'··· 37a8eec1ce19687d132fe29051dca629d164e2c4958ba141d5f4133a33f0688f root@mq-228 ~/i/chdb# echo -n \"tom-password\" | sha256sum | tr -d '-' d8c862690b30f6f2add244327715cb08ac926c7c2fb4fcbb7694650bfde5b672 default用户密码为default，tom的密码为tom-password，添加至users.xml，在 users/default中删除password项，添加： \u003cpassword_sha256_hex\u003e37a8eec1ce19687d132fe29051dca629d164e2c4958ba141d5f4133a33f0688f\u003c/password_sha256_hex\u003e 在users/下添加： \u003ctom\u003e \u003cpassword_sha256_hex\u003ed8c862690b30f6f2add244327715cb08ac926c7c2fb4fcbb7694650bfde5b672\u003c/password_sha256_hex\u003e \u003cprofile\u003edefault\u003c/profile\u003e \u003cquota\u003edefault\u003c/quota\u003e \u003cnetworks incl=\"networks\" replace=\"replace\"\u003e \u003cip\u003e::/0\u003c/ip\u003e \u003c/networks\u003e \u003c/tom\u003e 随后即可通过以下命令登陆数据库： clickhouse-client -u default --password default --port 9001 clickhouse-client -u tom --password tom-password --port 9001 ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:4:2","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#添加用户和密码"},{"categories":["时序数据库"],"collections":null,"content":"设置监听网段 confix.xml中取消 listen_host 行的注释： \u003c!-- Listen specified host. use :: (wildcard IPv6 address), if you want to accept connections both with IPv4 and IPv6 from everywhere. --\u003e \u003clisten_host\u003e::\u003c/listen_host\u003e 数据库文件主要由 /etc/clickhouse-server/users.xml 、/etc/clickhouse-server/config.xml 、/etc/metrika.xml 文件构成，其中metrika.xml的内容会覆盖config.xml对应内容，由用户自行创建，主要用于设置分片和备份。metrika.xml的文件名和路径可以在config.xml中自定义。 ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:4:3","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#设置监听网段"},{"categories":["时序数据库"],"collections":null,"content":"s1服务器 工作目录下的文件列表参考： root@mq-227 ~/i/chdb# tree . . ├── check_table_p.sh ├── config-s1r1.xml ├── config-s3r2.xml ├── create_table.sh ├── data_p.csv ├── delete_table_p.sh ├── docker-compose.yaml ├── metrika-s1r1.xml ├── metrika-s3r2.xml ├── metrika.xml ├── query_table.sh └── users.xml metrika-s1r1.xml内容（稍后在docker-compose.xml中会映射到容器内的metrika.xml）： \u003cyandex\u003e \u003c!-- 集群配置 --\u003e \u003cclickhouse_remote_servers\u003e \u003c!-- 3分片2备份，perftest_3shards_2replicas 为唯一ID--\u003e \u003cperftest_3shards_2replicas\u003e \u003cshard\u003e \u003cinternal_replication\u003efalse\u003c/internal_replication\u003e \u003creplica\u003e \u003chost\u003es1r1\u003c/host\u003e \u003cport\u003e9001\u003c/port\u003e \u003cuser\u003edefault\u003c/user\u003e \u003cpassword\u003edefault\u003c/password\u003e \u003c/replica\u003e \u003creplica\u003e \u003chost\u003es1r2\u003c/host\u003e \u003cport\u003e9002\u003c/port\u003e \u003cuser\u003edefault\u003c/user\u003e \u003cpassword\u003edefault\u003c/password\u003e \u003c/replica\u003e \u003c/shard\u003e \u003cshard\u003e \u003cinternal_replication\u003efalse\u003c/internal_replication\u003e \u003creplica\u003e \u003chost\u003es2r1\u003c/host\u003e \u003cport\u003e9001\u003c/port\u003e \u003cuser\u003edefault\u003c/user\u003e \u003cpassword\u003edefault\u003c/password\u003e \u003c/replica\u003e \u003creplica\u003e \u003chost\u003es2r2\u003c/host\u003e \u003cport\u003e9002\u003c/port\u003e \u003cuser\u003edefault\u003c/user\u003e \u003cpassword\u003edefault\u003c/password\u003e \u003c/replica\u003e \u003c/shard\u003e \u003cshard\u003e \u003cinternal_replication\u003efalse\u003c/internal_replication\u003e \u003creplica\u003e \u003chost\u003es3r1\u003c/host\u003e \u003cport\u003e9001\u003c/port\u003e \u003cuser\u003edefault\u003c/user\u003e \u003cpassword\u003edefault\u003c/password\u003e \u003c/replica\u003e \u003creplica\u003e \u003chost\u003es3r2\u003c/host\u003e \u003cport\u003e9002\u003c/port\u003e \u003cuser\u003edefault\u003c/user\u003e \u003cpassword\u003edefault\u003c/password\u003e \u003c/replica\u003e \u003c/shard\u003e \u003c/perftest_3shards_2replicas\u003e \u003c/clickhouse_remote_servers\u003e \u003c!-- ZooKeeper 配置 --\u003e \u003czookeeper-servers\u003e \u003cnode index=\"1\"\u003e \u003chost\u003ezoo1\u003c/host\u003e \u003cport\u003e2181\u003c/port\u003e \u003c/node\u003e \u003cnode index=\"2\"\u003e \u003chost\u003ezoo2\u003c/host\u003e \u003cport\u003e2181\u003c/port\u003e \u003c/node\u003e \u003cnode index=\"3\"\u003e \u003chost\u003ezoo3\u003c/host\u003e \u003cport\u003e2181\u003c/port\u003e \u003c/node\u003e \u003csession_timeout_ms\u003e30000\u003c/session_timeout_ms\u003e \u003coperation_timeout_ms\u003e10000\u003c/operation_timeout_ms\u003e \u003c/zookeeper-servers\u003e \u003c!-- 环境变量，不同分片需要替换ID，在创建表时可以引用该变量从而实现建表语句统一--\u003e \u003cmacros\u003e \u003cshard\u003es1\u003c/shard\u003e \u003creplica\u003er1\u003c/replica\u003e \u003c/macros\u003e \u003c/yandex\u003e 复制之前导出的config.xml并重命名为config-s1r1.xml （后文同），修改其TCP连接端口和同步端口： \u003ctcp_port\u003e9001\u003c/tcp_port\u003e \u003cinterserver_http_port\u003e9011\u003c/interserver_http_port\u003e metrika-s3r2.xml 与 metrika-s1r1.xml 内容相同，只需修改环境变量配置部分： \u003cmacros\u003e \u003cshard\u003es3\u003c/shard\u003e \u003creplica\u003er2\u003c/replica\u003e \u003c/macros\u003e config-s3r2.xml 修改其TCP连接端口和同步端口: \u003ctcp_port\u003e9002\u003c/tcp_port\u003e \u003cinterserver_http_port\u003e9013\u003c/interserver_http_port\u003e docker-compose.xml内容： version: '3.1' services: chdb-s1r1: image: yandex/clickhouse-server:latest hostname: s1r1 ports: - 9001:9001 - 9011:9011 volumes: - /root/iot/chdb/users.xml:/etc/clickhouse-server/users.xml - /root/iot/chdb/config-s1r1.xml:/etc/clickhouse-server/config.xml - /root/iot/chdb/metrika-s1r1.xml:/etc/metrika.xml - /root/iot/chdb/s1r1:/var/lib/clickhouse extra_hosts: - \"s1r1:[S1]\" - \"s1r2:[S2]\" - \"s2r1:[S2]\" - \"s2r2:[S3]\" - \"s3r1:[S3]\" - \"s3r2:[S1]\" - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"zoo3:[S3]\" chdb-s3r2: image: yandex/clickhouse-server:latest hostname: s3r2 ports: - 9002:9002 - 9013:9013 volumes: - /root/iot/chdb/users.xml:/etc/clickhouse-server/users.xml - /root/iot/chdb/config-s3r2.xml:/etc/clickhouse-server/config.xml - /root/iot/chdb/metrika-s3r2.xml:/etc/metrika.xml - /root/iot/chdb/s3r2:/var/lib/clickhouse extra_hosts: - \"s1r1:[S1]\" - \"s1r2:[S2]\" - \"s2r1:[S2]\" - \"s2r2:[S3]\" - \"s3r1:[S3]\" - \"s3r2:[S1]\" - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"zoo3:[S3]\" volumes选项中的本地路径需要按实际部署路径修改（下同），其中最后一条为数据库数据本地持久化映射，可按需更改文件位置。 ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:4:4","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#s1服务器"},{"categories":["时序数据库"],"collections":null,"content":"s2服务器 metrika-s2r1.xml 与 metrika-s1r1.xml 类似，只需修改环境变量配置部分： \u003cmacros\u003e \u003cshard\u003es2\u003c/shard\u003e \u003creplica\u003er1\u003c/replica\u003e \u003c/macros\u003e config-s2r1.xml 修改其TCP连接端口和同步端口: \u003ctcp_port\u003e9001\u003c/tcp_port\u003e \u003cinterserver_http_port\u003e9012\u003c/interserver_http_port\u003e metrika-s1r2.xml 与 metrika-s1r1.xml 类似，只需修改环境变量配置部分： \u003cmacros\u003e \u003cshard\u003es1\u003c/shard\u003e \u003creplica\u003er2\u003c/replica\u003e \u003c/macros\u003e config-s1r2.xml 修改其TCP连接端口和同步端口: \u003ctcp_port\u003e9002\u003c/tcp_port\u003e \u003cinterserver_http_port\u003e9011\u003c/interserver_http_port\u003e docker-compose.xml内容： version: '3.1' services: chdb-s2r1: image: yandex/clickhouse-server:latest hostname: s2r1 ports: - 9001:9001 - 9012:9012 volumes: - /root/iot/chdb/users.xml:/etc/clickhouse-server/users.xml - /root/iot/chdb/config-s2r1.xml:/etc/clickhouse-server/config.xml - /root/iot/chdb/metrika-s2r1.xml:/etc/metrika.xml - /root/iot/chdb/s2r1:/var/lib/clickhouse extra_hosts: - \"s1r1:[S1]\" - \"s1r2:[S2]\" - \"s2r1:[S2]\" - \"s2r2:[S3]\" - \"s3r1:[S3]\" - \"s3r2:[S1]\" - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"zoo3:[S3]\" chdb-s1r2: image: yandex/clickhouse-server:latest hostname: s1r2 ports: - 9002:9002 - 9011:9011 volumes: - /root/iot/chdb/users.xml:/etc/clickhouse-server/users.xml - /root/iot/chdb/config-s1r2.xml:/etc/clickhouse-server/config.xml - /root/iot/chdb/metrika-s1r2.xml:/etc/metrika.xml - /root/iot/chdb/s1r2:/var/lib/clickhouse extra_hosts: - \"s1r1:[S1]\" - \"s1r2:[S2]\" - \"s2r1:[S2]\" - \"s2r2:[S3]\" - \"s3r1:[S3]\" - \"s3r2:[S1]\" - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"zoo3:[S3]\" ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:4:5","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#s2服务器"},{"categories":["时序数据库"],"collections":null,"content":"s3服务器 metrika-s3r1.xml 需修改环境变量配置部分： \u003cmacros\u003e \u003cshard\u003es3\u003c/shard\u003e \u003creplica\u003er1\u003c/replica\u003e \u003c/macros\u003e config-s3r1.xml 修改其TCP连接端口和同步端口: \u003ctcp_port\u003e9001\u003c/tcp_port\u003e \u003cinterserver_http_port\u003e9013\u003c/interserver_http_port\u003e metrika-s2r2.xml 修改环境变量配置部分： \u003cmacros\u003e \u003cshard\u003es2\u003c/shard\u003e \u003creplica\u003er2\u003c/replica\u003e \u003c/macros\u003e config-s2r2.xml 修改其TCP连接端口和同步端口: \u003ctcp_port\u003e9002\u003c/tcp_port\u003e \u003cinterserver_http_port\u003e9012\u003c/interserver_http_port\u003e docker-compose.xml内容： version: '3.1' services: chdb-s3r1: image: yandex/clickhouse-server:latest hostname: s3r1 ports: - 9001:9001 - 9013:9013 volumes: - /root/iot/chdb/users.xml:/etc/clickhouse-server/users.xml - /root/iot/chdb/config-s3r1.xml:/etc/clickhouse-server/config.xml - /root/iot/chdb/metrika-s3r1.xml:/etc/metrika.xml - /root/iot/chdb/s3r1:/var/lib/clickhouse extra_hosts: - \"s1r1:[S1]\" - \"s1r2:[S2]\" - \"s2r1:[S2]\" - \"s2r2:[S3]\" - \"s3r1:[S3]\" - \"s3r2:[S1]\" - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"zoo3:[S3]\" chdb-s2r2: image: yandex/clickhouse-server:latest hostname: s2r2 ports: - 9002:9002 - 9012:9012 volumes: - /root/iot/chdb/users.xml:/etc/clickhouse-server/users.xml - /root/iot/chdb/config-s2r2.xml:/etc/clickhouse-server/config.xml - /root/iot/chdb/metrika-s2r2.xml:/etc/metrika.xml - /root/iot/chdb/s2r2:/var/lib/clickhouse extra_hosts: - \"s1r1:[S1]\" - \"s1r2:[S2]\" - \"s2r1:[S2]\" - \"s2r2:[S3]\" - \"s3r1:[S3]\" - \"s3r2:[S1]\" - \"zoo1:[S1]\" - \"zoo2:[S2]\" - \"zoo3:[S3]\" ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:4:6","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#s3服务器"},{"categories":["时序数据库"],"collections":null,"content":"验证 连接至任意数据库，查询集群信息： clickhouse-client -u default --password default --host s1r1 --port 9001 ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:5:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#验证"},{"categories":["时序数据库"],"collections":null,"content":"示例 ClickHouse备份使用的是表级备份，要使用备份需要使用 Replicated*表引擎，需要注意的是ClickHouse集群在执行 CREATE、 DROP、ATTACH、DETACH和RENAME时不会同步命令，因此创建表时需要在每个数据库实例中都进行创建。 在s1服务器上执行s1/create_table_p.sh脚本来创建表： ports=\"9001 9002\" hosts=\"s1 s2 s3\" for port in $ports do for host in $hosts do echo \"Creating table on $host:$port\" clickhouse-client -u default --password default --host $host --port $port --query \\ \"CREATE TABLE p ( ozone Int8, particullate_matter Int8, carbon_monoxide Int8, sulfure_dioxide Int8, nitrogen_dioxide Int8, longitude Float64, latitude Float64, timestamp DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/p/{shard}','{replica}') ORDER BY timestamp PRIMARY KEY timestamp\" done done 为了使用分片，需要一个虚拟的统一入口表，ClickHouse中称为分布表（Distributed Table），可以简单认为是一个数据路由表。创建分布表： clickhouse-client --host s1r1 -u default --password default --port 9001 --query \"CREATE TABLE p_all AS p ENGINE = Distributed(perftest_3shards_2replicas, default, p, rand())\" 其中 perftest_3shards_2replicas 为之前定义的集群ID。然后导入测试数据： clickhouse-client --host s1r1 -u default --password default --port 9001 --query \"INSERT INTO p_all FORMAT CSV\" \u003c data_p.csv 查看数据分片存储的情况 s1/check_table_p.sh： ports=\"9001 9002\" hosts=\"s1 s2 s3\" for port in $ports do for host in $hosts do echo \"Data from $host:$port\" clickhouse-client -u default --password default --host $host --port $port --query \"select count(*) from p\" done done ","date":"2020-08-31","objectID":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/:6:0","tags":["ClickHouse","时序数据库"],"title":"ClickHouse 多分片多备份部署","uri":"/posts/clickhouse-%E5%A4%9A%E5%88%86%E7%89%87%E5%A4%9A%E5%A4%87%E4%BB%BD%E9%83%A8%E7%BD%B2/#示例"},{"categories":["Java"],"collections":null,"content":"JavaFX系列","date":"2020-05-04","objectID":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/","tags":["JavaFX","GUI"],"title":"JavaFX系列-开发环境","uri":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/"},{"categories":["Java"],"collections":null,"content":"本文主要介绍JavaFX环境的搭建和如何部署JavaFX应用 ","date":"2020-05-04","objectID":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/:0:0","tags":["JavaFX","GUI"],"title":"JavaFX系列-开发环境","uri":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/#"},{"categories":["Java"],"collections":null,"content":"添加JavaFX依赖 在工程 build.gradle 中加入以下内容： plugins { id 'application' id 'org.openjfx.javafxplugin' version '0.0.8' // 引入JavaFX的jar包 } javafx { version = \"14\" // Java版本 modules = [ 'javafx.controls' ] // 所需要的模块 manifest { mainClassName = 'com.ingbyr.vdm.gui.Main' // 运行入口 } } ","date":"2020-05-04","objectID":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/:1:0","tags":["JavaFX","GUI"],"title":"JavaFX系列-开发环境","uri":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/#添加javafx依赖"},{"categories":["Java"],"collections":null,"content":"自定义运行时 Java9引入的Jigsaw允许我们制作自己的JRE，从而有效地减少应用的大小 首先下载JavaFX的jmods保存到 ./jmods/中 jmods ├── javafx.base.jmod ├── javafx.controls.jmod ├── javafx.fxml.jmod ├── javafx.graphics.jmod ├── javafx.media.jmod ├── javafx.swing.jmod └── javafx.web.jmod 然后使用jlink生成自定义运行时 jlink \\ --module-path ./jmods \\ --add-modules java.base,javafx.base,javafx.graphics,javafx.controls \\ --output runtime module-path 第三方模块的目录 add-modules 运行时所需的模块 output 自定义运行时目录名称 生成的运行时目录如下 runtime ├── bin ├── conf ├── include ├── legal ├── lib ├── man └── release ","date":"2020-05-04","objectID":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/:2:0","tags":["JavaFX","GUI"],"title":"JavaFX系列-开发环境","uri":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/#自定义运行时"},{"categories":["Java"],"collections":null,"content":"编译应用 使用gradle插件application中提供的tasks -\u003e distribution -\u003e installDist构建任务来编译应用，生成的目录如下 build/install └── vdm-gui ├── bin │ ├── vdm-gui │ └── vdm-gui.bat └── lib ├── javafx-base-14-mac.jar ├── javafx-base-14.jar ├── javafx-controls-14-mac.jar ├── javafx-graphics-14-mac.jar ├── javafx-graphics-14.jar └── vdm-gui-2.0.0.jar ","date":"2020-05-04","objectID":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/:3:0","tags":["JavaFX","GUI"],"title":"JavaFX系列-开发环境","uri":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/#编译应用"},{"categories":["Java"],"collections":null,"content":"部署应用 Java14中正式提供了jpackage工具，该工具可以十分方便的部署跨平台应用安装包，示例如下 jpackage \\ --input vdm-gui/build/install/vdm-gui/lib \\ --name vdm14 \\ --main-jar vdm-gui-2.0.0.jar \\ --main-class com.ingbyr.vdm.gui.Main \\ --type dmg \\ --runtime-image ~/IdeaProjects/vdm14/runtime input 需要打包的jar目录 name 应用名称 main-jar 主程序jar包 main-class 若jar中没有指定主类，则可以使用此参数指定 type 安装包类型，可选： Windows: exe msi Linux: rpm deb MacOS: pkg dmg runtime-image 自定义的运行时 ","date":"2020-05-04","objectID":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/:4:0","tags":["JavaFX","GUI"],"title":"JavaFX系列-开发环境","uri":"/posts/javafx%E7%B3%BB%E5%88%97-%E5%BC%80%E5%8F%91%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/#部署应用"},{"categories":null,"collections":null,"content":"Spigot服务器（类Bukkit） ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:1:0","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#spigot服务器类bukkit"},{"categories":null,"collections":null,"content":"介绍 spigotmc是一个以高性能著称的MC服务器端，Spigot服务端只支持plugin形式的扩展，如果使用人数规模庞大需要追求高性能服务器，Spigot是不二的选择。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:1:1","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#介绍"},{"categories":null,"collections":null,"content":"安装 在官网下载BuildTools.jar文件后，需要自行编译出所需版本的服务器端文件，格式如下： java -jar BuildTools.jar --rev [版本号] 编译需要一段时间，完成编译后将生成的jar文件拷贝至单独的文件夹。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:1:2","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#安装"},{"categories":null,"collections":null,"content":"插件配置 将插件（支持Spigot插件和Bukkit插件）拖入plugins文件夹中即可。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:1:3","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#插件配置"},{"categories":null,"collections":null,"content":"运行服务器 在jar文件目录下，编写一小段脚本以简化启动命令，内容如下： java -Xms1024M -Xmx1024M -jar [your-jar-file] nogui 启动一个Screen终端，然后输入 sh start.sh 等待服务器启动完毕即可，要使服务器后台运行，按下ctrl+a,d后screen将处于后台。恢复screen输入： screen -r mc 终止服务器直接 ctrl+c 即可 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:1:4","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#运行服务器"},{"categories":null,"collections":null,"content":"Forge服务器 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:2:0","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#forge服务器"},{"categories":null,"collections":null,"content":"介绍 如果服务端需要运行一些mod（例如著名的暮色森林mod），此时spigot就不再适用了，而Forge正是为mod而生的服务端和客户端工具。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:2:1","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#介绍-1"},{"categories":null,"collections":null,"content":"安装 在Forge下载页面选择需要的forge版本，生成服务端所需的文件要在本地新建一个空文件夹，生成所需文件后整体上传到服务器。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:2:2","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#安装-1"},{"categories":null,"collections":null,"content":"mod配置 将mod直接拖入mod文件夹即可。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:2:3","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#mod配置"},{"categories":null,"collections":null,"content":"运行服务器 运行forge*.jar文件即可创建服务器。具体可参考Spigot中的 运行服务器一节 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:2:4","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#运行服务器-1"},{"categories":null,"collections":null,"content":"Sponge ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:3:0","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#sponge"},{"categories":null,"collections":null,"content":"介绍 Sponge据说是同时支持Bukkit插件和MOD功能的服务器端工具，具体没用使用过，大家可以去官网自行阅读文档安装体验。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:3:1","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#介绍-2"},{"categories":null,"collections":null,"content":"MC客户端 推荐使用HMCL。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:4:0","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#mc客户端"},{"categories":null,"collections":null,"content":"安装optifine 虽然HMCL中有自动安装optifine的功能，但经常会出现依赖配置错误导致的运行失败，所以optifine和forge推荐自行安装。选择对应版本下载optifine，之后将jar文件导入mod或使用HMCL提供的MOD管理导入即可。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:4:1","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#安装optifine"},{"categories":null,"collections":null,"content":"安装forge 在Forge下载页面选择需要的forge版本，安装客户端forge时需要选中游戏所在的.minecraft文件夹（不是公共资源文件夹），然后点击安装即可。 ","date":"2018-08-03","objectID":"/posts/mc-linux-server/:4:2","tags":null,"title":"Minecraft服务器搭建","uri":"/posts/mc-linux-server/#安装forge"},{"categories":null,"collections":null,"content":"样例 我们要达到的视频效果是这样的 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:1:0","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#样例"},{"categories":null,"collections":null,"content":"确定压缩目标 bilibili不进行二压的参数要求如下： 视频码率最高1800kbps（H264/AVC编码） 音频码率最高192kbps（AAC编码） 分辨率最大支持1920x1080 level≤4.1 关键帧平均至少10秒一个 色彩空间yuv420 位深8bit 声道数≤2 采样率=44100 （不满足条件的视频会被系统二次压制） ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:2:0","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#确定压缩目标"},{"categories":null,"collections":null,"content":"FFMPEG压制 这里先给出全部命令： ffmpeg -i $argv[1] -vcodec libx264 -preset veryslow -profile:v high -level:v 4.1 -pix_fmt yuv420p -b:v 1780k -r 60 -acodec aac -strict -2 -ac 2 -ab 128k -ar 44100 -pass 1 -f flv /dev/null; and ffmpeg -i $argv[1] -vcodec libx264 -preset veryslow -profile:v high -level:v 4.1 -pix_fmt yuv420p -b:v 1780k -r 60 -acodec aac -strict -2 -ac 2 -ab 128k -ar 44100 -pass 2 -f flv $argv[2] 这里总共执行了两条命令，下面详细说一下。 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:0","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#ffmpeg压制"},{"categories":null,"collections":null,"content":"-i $argv[1] 这里指定了目标视频路径，也就是要处理的视频文件 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:1","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-i-argv1"},{"categories":null,"collections":null,"content":"-vcodec libx264 使用X264编码器 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:2","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-vcodec-libx264"},{"categories":null,"collections":null,"content":"-preset veryslow 使用h.264的最佳编码，牺牲了编码速度。因为b站1800的码率如果不采用最佳编码，会导致画面极度模糊。 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:3","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-preset-veryslow"},{"categories":null,"collections":null,"content":"-profile:v high -level:v 4.1 设备兼容性，这里不需要修改 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:4","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-profilev-high--levelv-41"},{"categories":null,"collections":null,"content":"-pix_fmt yuv420p 色彩空间yuv420p，b站要求 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:5","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-pix_fmt-yuv420p"},{"categories":null,"collections":null,"content":"-b:v 1780k 码率采用1780，如果采用上限1800实际结果将有个可能超过这个值，从而被二压 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:6","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-bv-1780k"},{"categories":null,"collections":null,"content":"-r 60 视频帧率为60FPS ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:7","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-r-60"},{"categories":null,"collections":null,"content":"-pass 1 说明当前处理为第一次处理，为了达到稳定的视频目标参数我们需要进行两次压制，第二条命令就是第二次压制 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:8","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-pass-1"},{"categories":null,"collections":null,"content":"acodec aac -strict -2 -ac 2 -ab 128k -ar 44100 音频参数，说明使用acc解码器，双声道，128K码率，44.1k采样率，都是b站的上限数值 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:9","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#acodec-aac--strict--2--ac-2--ab-128k--ar-44100"},{"categories":null,"collections":null,"content":"-f flv 视频格式为flv ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:3:10","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#-f-flv"},{"categories":null,"collections":null,"content":"上传视频 经过本地的两次压制，上传到b站后就不会被二压，从而保证了60fps的帧率。 ","date":"2017-12-19","objectID":"/posts/compress-video-by-ffmpeg/:4:0","tags":["Tutorial"],"title":"使用ffmpeg压缩60FPS视频并上传至Bilibili","uri":"/posts/compress-video-by-ffmpeg/#上传视频"},{"categories":null,"collections":null,"content":"啰嗦几句 权利的游戏终于又更新啦，然而目前网络没有IPV6所以怎么下载资源成了最大的问题，试了试公网上的bt，速度慢的令人发指，所以决定还是想办法通过IPV6下载资源 ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:1:0","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#啰嗦几句"},{"categories":null,"collections":null,"content":"需要提前准备的资源 必须要有的： 一台能访问IPV6资源的服务器（本文使用一台Ubuntu Server 16.04） 一个BT账号（北邮人应该都有吧） 如果是校内服务器，还需要下载GlobalProtect，后续步骤默认开启VPN 以下资源是非必需的： 独立域名（方便自己记住域名） ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:2:0","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#需要提前准备的资源"},{"categories":null,"collections":null,"content":"步骤 ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:0","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#步骤"},{"categories":null,"collections":null,"content":"安装Deluge Deluge后端使用libtorrent，可以在多个平台上使用 sudo apt install deluge ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:1","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#安装deluge"},{"categories":null,"collections":null,"content":"安装Deluge-web Deluge-web可以将Deluge在浏览器中以可视化的方式展现出来，应该没人希望通过复杂的命令行方式操作吧 sudo apt install deluge-webui ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:2","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#安装deluge-web"},{"categories":null,"collections":null,"content":"运行Deluge 启动Deluge后还需要手动启动Deluge-webui，这里使用screen启动webui sudo systemctl start deluged screen -S deluge deluge-web 这时在浏览器中访问 http://your-server-ip:8112 就可以看到熟悉的bt下载界面了，默认的密码是deluge ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:3","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#运行deluge"},{"categories":null,"collections":null,"content":"懒人必备（非必须步骤） 一串数字的网址毕竟不太好记忆，况且还有一个8112的端口号，所以接下来使用dnspod和nginx实现域名访问 安装nginx sudo apt install nginx 创建配置文件 在 /etc/nginx/conf.d/ 目录下新建一个配置文件，名称随意后缀为.conf，针对8112端口的配置实例如下： server { listen 80; server_name [你的域名]; location / { proxy_redirect off; proxy_pass http://127.0.0.1:8112; } } 如果要配置其他端口的照猫画虎再填一个就行了，因为在 /etc/nginx/nginx.conf 默认读入了conf.d文件夹下的配置文件，所以不再需要动它的默认配置文件 重启nginx sudo systemctl restart nginx.service 绑定域名 在DNS服务商添加一条A记录，域名为上文中的域名，ip指向服务器IP即可，等DNS生效就可以通过二级域名访问Deluge了 ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:4","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#懒人必备非必须步骤"},{"categories":null,"collections":null,"content":"懒人必备（非必须步骤） 一串数字的网址毕竟不太好记忆，况且还有一个8112的端口号，所以接下来使用dnspod和nginx实现域名访问 安装nginx sudo apt install nginx 创建配置文件 在 /etc/nginx/conf.d/ 目录下新建一个配置文件，名称随意后缀为.conf，针对8112端口的配置实例如下： server { listen 80; server_name [你的域名]; location / { proxy_redirect off; proxy_pass http://127.0.0.1:8112; } } 如果要配置其他端口的照猫画虎再填一个就行了，因为在 /etc/nginx/nginx.conf 默认读入了conf.d文件夹下的配置文件，所以不再需要动它的默认配置文件 重启nginx sudo systemctl restart nginx.service 绑定域名 在DNS服务商添加一条A记录，域名为上文中的域名，ip指向服务器IP即可，等DNS生效就可以通过二级域名访问Deluge了 ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:4","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#安装nginx"},{"categories":null,"collections":null,"content":"懒人必备（非必须步骤） 一串数字的网址毕竟不太好记忆，况且还有一个8112的端口号，所以接下来使用dnspod和nginx实现域名访问 安装nginx sudo apt install nginx 创建配置文件 在 /etc/nginx/conf.d/ 目录下新建一个配置文件，名称随意后缀为.conf，针对8112端口的配置实例如下： server { listen 80; server_name [你的域名]; location / { proxy_redirect off; proxy_pass http://127.0.0.1:8112; } } 如果要配置其他端口的照猫画虎再填一个就行了，因为在 /etc/nginx/nginx.conf 默认读入了conf.d文件夹下的配置文件，所以不再需要动它的默认配置文件 重启nginx sudo systemctl restart nginx.service 绑定域名 在DNS服务商添加一条A记录，域名为上文中的域名，ip指向服务器IP即可，等DNS生效就可以通过二级域名访问Deluge了 ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:4","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#创建配置文件"},{"categories":null,"collections":null,"content":"懒人必备（非必须步骤） 一串数字的网址毕竟不太好记忆，况且还有一个8112的端口号，所以接下来使用dnspod和nginx实现域名访问 安装nginx sudo apt install nginx 创建配置文件 在 /etc/nginx/conf.d/ 目录下新建一个配置文件，名称随意后缀为.conf，针对8112端口的配置实例如下： server { listen 80; server_name [你的域名]; location / { proxy_redirect off; proxy_pass http://127.0.0.1:8112; } } 如果要配置其他端口的照猫画虎再填一个就行了，因为在 /etc/nginx/nginx.conf 默认读入了conf.d文件夹下的配置文件，所以不再需要动它的默认配置文件 重启nginx sudo systemctl restart nginx.service 绑定域名 在DNS服务商添加一条A记录，域名为上文中的域名，ip指向服务器IP即可，等DNS生效就可以通过二级域名访问Deluge了 ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:4","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#重启nginx"},{"categories":null,"collections":null,"content":"懒人必备（非必须步骤） 一串数字的网址毕竟不太好记忆，况且还有一个8112的端口号，所以接下来使用dnspod和nginx实现域名访问 安装nginx sudo apt install nginx 创建配置文件 在 /etc/nginx/conf.d/ 目录下新建一个配置文件，名称随意后缀为.conf，针对8112端口的配置实例如下： server { listen 80; server_name [你的域名]; location / { proxy_redirect off; proxy_pass http://127.0.0.1:8112; } } 如果要配置其他端口的照猫画虎再填一个就行了，因为在 /etc/nginx/nginx.conf 默认读入了conf.d文件夹下的配置文件，所以不再需要动它的默认配置文件 重启nginx sudo systemctl restart nginx.service 绑定域名 在DNS服务商添加一条A记录，域名为上文中的域名，ip指向服务器IP即可，等DNS生效就可以通过二级域名访问Deluge了 ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:3:4","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#绑定域名"},{"categories":null,"collections":null,"content":"资源取回 可以选择把资源从服务器上下下来本地观看，也可以使用Potplayer等支持远程视频播放的软件直接观看。 ","date":"2017-08-14","objectID":"/posts/remote-torrent-sys/:4:0","tags":["Linux","Tools"],"title":"Linux Server使用IPV6 BT资源","uri":"/posts/remote-torrent-sys/#资源取回"},{"categories":null,"collections":null,"content":"微波通信 今天的实习内容为“微波通信”。这次讲课中刘老师带我们回顾了一个微波通信系统是如何工作的，演示了一个微波中继基站的实际工作方式，详细讲解了一些重点的相关理论，这些都让我有一种豁然开朗的感觉，个人认为对以后考研复习有很大帮助。下午的上课时间由同学们自由组队，进行了小组展示，展示的内容是QPSK、载波提取等通信系统中常见的基本理论。我们小组展示了线路编码的基本理论，让我对编码有了进一步的认识。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:1:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#微波通信"},{"categories":null,"collections":null,"content":"程控交换及运营模拟 今天的实习内容是“程控交换及模拟运营“。吴老师在课程开始时提到本次专业实习的目的是：树职业意识、培养职业素质、积累职业经验、掌握职业技能，刚开始听到时没有太大的感触，但老师随后演示了多种拨号方式，结合所学知识讲解了为什么可以用多种方式进行拨号。然后大家阅读了《沙盘模拟式实习指导手册》，为下午的情景模拟做准备。下午上课大家分为三组，分别模拟用户、服务人员、技术人员，就如何互相沟通、合作等展开了情景模拟，在将近1小时的模拟中，我深刻认识到做一个合格的运营商需要付出很多。本次实习获益匪浅。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:2:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#程控交换及运营模拟"},{"categories":null,"collections":null,"content":"计算机网络及运营模拟 今天的实习内容是“计算机网络及运营模拟”。首先吴老师让我们根据自己的理解画出了一个校园网组织架构图，然后通过一步一步的纠正让我们对校园网架构有了初步的了解，然后以校园网为出发点，大家分组自行设计一个实际的运行架构图并讨论其可行性。下午我们使用实验室的主机、交换机和路由器对上午的设想进行模拟验证。我们组希望搭建一个异地容灾备份的拓扑图，其中使用两台主机模拟服务器，都连接之一个交换机，然后通过路由器进行联网，第三台主机模拟资源请求。因为之前现代通信技术已经做过类似的组网实验，所以在搭建拓扑时没有遇到太大的困难，然而由于对ftp使用不熟练，没有成功搭建起来一个服务器，比较可惜。这次实验让我对组网有了更深次的理解。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:3:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#计算机网络及运营模拟"},{"categories":null,"collections":null,"content":"电声演播数字电视 今天的实习内容是“电视演播数字电视”。姜老师首先带领我们参观了一些上个世纪的多媒体设备，其中印象最深刻的就是一个音频处理设备。据老师所说，这套设备是联合国教科文组织当时提供资金给北邮进行建设的，83年版本的西游记就是在这里完成的后期配音。这一点就可以感受到当时的北邮实力相当雄厚。参观完设备后老师向我们讲解了多媒体通信中的一些相关理论知识，例如视频压缩编码等，结合之前相关理论课的知识，初步了解了多媒体通信中使用到的技术和要解决的问题。听完老师的讲解之后，我对多媒体有了新的认识。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:4:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#电声演播数字电视"},{"categories":null,"collections":null,"content":"光通信 今天的实习内容是“光通信”。老师首先讲解了ADM设备在光通信中的作用，ADM时一种多路低俗的电信号转换为一路高速的光信号设备，在光通信中占据着很重要的作用。然后使用了三台由华为生产的ADM进行了组网实验，将三台ADM进行环形链接，组成一个SDN环网，该网具有一定的容灾能力。随后老师讲解了光纤的熔接技术，由于光纤的弱导特性，需要对两段光纤链接进行特殊的处理。一种是采用冷接的技术，就是在光纤端口处加上一种连接口，好处是操作方便，缺点是在连接处损耗大（可以达到1db），所以在实际的大型工程中不常用。工程中使用的时热接技术，即熔接。在老师的指导下我们进行了两段光纤的熔接操作，在一系列步骤后成功将两段光纤进行了熔接。下午在一个光通信实验室参观了很多种光通信中实际使用的设备，随后在指导下分组进行了软件模拟实验和SDN软件定义网络的操作实验。本次实验让我第一次接触到了光通信，收货颇多。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:5:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#光通信"},{"categories":null,"collections":null,"content":"音视频处理 今天的实习内容是“音视频处理”。老师讲解了一些在现代多媒体通信中常见的几种手段，展示了一些编码算法等。然后又向我们讲解机器学习导论，姜老师通过讲解ppt向我们展示了机器学习、模式识别、统计识别、计算机视觉和语音识别等基础概念，老师以百度识图和siri生动的展示了这几个学科的实际应用情况。随后重点讲解了一些算法背景，如深度学习等。通过这次导论课，我了解到了机器学习在现在的社会中已经处于重要的地位，许多工程都离不开机器学习。我对机器学习也有了初步的了解，它是一门多领域交叉学科，涉及到概率论、统计学等等多种学科，主要目标时设计和分析一些让计算机可以自学习的算法。机器学习是现在的一个热门学科，它将会对人类发展起到不可或缺的贡献。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:6:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#音视频处理"},{"categories":null,"collections":null,"content":"天线、射频 今天的实习内容是“天线与射频”。老师讲了天线实验所需要的试验环境和一些基本注意事项，实验环境原本是要求一个暗室，但由于学校实验室建设受到的限制无法满足，所以实验作为定性实验来演示。演示实验包括两个，一个是电路方面的测量天线输入阻抗，另一个是辐射方面的天线方向图。第一个天线输入组坑测量的基本原理是以史密斯圆图为坐标系，使用网络分析仪来进行测量，在测量之前有很重要的一步，即使用校准件校准仪器，校准完毕后老师测量了三个不同天线的册数。在天线方向图的实验开始时，老师介绍了本次实验中使用的八木天线，因为这种天线往往使用了较多的引向器，所以看起来像是鱼骨而得名。因为八木天线具有增益高、方向性强、结构简单的优点，它被广泛应用在无线电测向和长距离无线电通信。而是描述天线或其它信号源发出无线电波的强度与方向（角度）之间依赖关系的图形。这次实验对天线参数的测量有了基本的了解。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:7:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#天线射频"},{"categories":null,"collections":null,"content":"网优网规路测 今天的实习内容是“网优网规路测”。老师提到了TD-LTE覆盖优化的问题。TD-LTE网络一般时同频组网，同频干扰严重，所以良好的覆盖和干扰控制对网络性能意义重大。覆盖优化主要是消除网络中存在的四中问题：覆盖空洞、弱覆盖、越区覆盖和导频污染，优化的目标有RSPR、RSRQ等指标。介绍完毕后老师给每个小组分发了手持式无线网络测试终端，每个小组在室外进行LTE信号的测试。我们组围绕着主楼和操场进行了500次ping操作。下午上课的时候将数据导入电脑中，使用分析软件进行了数据分析，并且将结果以总结报告的形式导出，简要分析了报告中的参数指标和其他内容。对于报告中暴漏的缺陷或问题，可以通过覆盖优化进行解决。常见的手段有调整天线下倾角、调整天线方位角、调整RS功率、新增站点等等。老师在实验过程中提供了很多的帮助。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:8:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#网优网规路测"},{"categories":null,"collections":null,"content":"移动通信 今天的实习内容是“移动通信”。老师重点讲解了TD-LTE 的基本原理和现状。TD-LTE的出现，是因为3G制式：WCDMA、CDMA2000和TD-SCDMA不兼容，所以要融合成为LTE。LTE的突出特点是宽带高、系统架构层数比以前架构更少，所以延迟较低。老师还介绍了MIMO技术，即多天线技术，MIMO的核心概念为利用多根发射天线与多根接收天线所提供之空间自由度来有效提升无线通信系统之频谱效率，以提升传输速率并改善通信质量。由于MIMO可以在不需要增加带宽或总发送功率耗损的情况下大幅地增加系统的数据吞吐量及发送距离，使得此技术于近几年受到许多瞩目。随后老师通过配置实验室两台RRU和手持终端，让我们体验了在不同参数配置下的TD-LTE服务质量。其中通过改变编码方式，可以在FTP测试中明显感受到下行带宽的巨大变化。这次实习让我深入的了解了现在TD-LTE的基本原理和发展状况。 ","date":"2016-07-04","objectID":"/posts/campus-internship-report/:9:0","tags":["Summary"],"title":"大三校内实习报告","uri":"/posts/campus-internship-report/#移动通信"},{"categories":["doc"],"collections":null,"content":"北邮校园网网关登陆工具，适配新网关 ngw.bupt.edu.cn。已开源至 GitHub。 ","date":"2016-04-14","objectID":"/posts/bupt-net-login/:0:0","tags":["Python","Tools"],"title":"北邮网关登陆工具","uri":"/posts/bupt-net-login/#"},{"categories":["doc"],"collections":null,"content":"安装 使用pip安装 pip3 install BUPTNetLogin 使用便携版 下载页面 ","date":"2016-04-14","objectID":"/posts/bupt-net-login/:1:0","tags":["Python","Tools"],"title":"北邮网关登陆工具","uri":"/posts/bupt-net-login/#安装"},{"categories":["doc"],"collections":null,"content":"使用方法 使用命令 bnl user@server ~\u003e bnl usage: bnl [-h] [-l {dx,xyw,lt,yd}] [-u USERNAME] [-p PASSWORD] [-lo] [-v] 北邮校园网网关登陆工具 optional arguments: -h, --help show this help message and exit -l {dx,xyw,lt,yd}, --login {dx,xyw,lt,yd} 登陆北邮校园网网关，LINE可用参数 xyw（校园网）、lt（联通）、yd（移动）、dx(电信) -u USERNAME, --username USERNAME 校园网账户名称 -p PASSWORD, --password PASSWORD 校园网账户密码 -lo, --logout 注销北邮校园网网关 -v, --version 版本信息 登陆联通网络举例： user@server ~\u003e bnl -l lt -u 用户名 -p 密码 注销网络举例 user@server ~\u003e bnl -lo ","date":"2016-04-14","objectID":"/posts/bupt-net-login/:2:0","tags":["Python","Tools"],"title":"北邮网关登陆工具","uri":"/posts/bupt-net-login/#使用方法"},{"categories":["doc"],"collections":null,"content":"更新 pip3 install BUPTNetLogin --upgrade ","date":"2016-04-14","objectID":"/posts/bupt-net-login/:3:0","tags":["Python","Tools"],"title":"北邮网关登陆工具","uri":"/posts/bupt-net-login/#更新"},{"categories":["doc"],"collections":null,"content":"依赖库 使用pip将自动安装以下库： requests lxml ","date":"2016-04-14","objectID":"/posts/bupt-net-login/:4:0","tags":["Python","Tools"],"title":"北邮网关登陆工具","uri":"/posts/bupt-net-login/#依赖库"},{"categories":["doc"],"collections":null,"content":"自行编译便携版 win: .\\venv\\Scripts\\pyinstaller -n bnl --noupx -c -F .\\app\\bupt_net_login.py linux: ./venv/bin/pyinstaller -n bnl --noupx -c -F ./app/bupt_net_login.py ","date":"2016-04-14","objectID":"/posts/bupt-net-login/:5:0","tags":["Python","Tools"],"title":"北邮网关登陆工具","uri":"/posts/bupt-net-login/#自行编译便携版"},{"categories":null,"collections":null,"content":" 微芯2016北邮奖学金获奖作品 ","date":"2015-11-21","objectID":"/posts/pic16f877-ljt/:0:0","tags":null,"title":"基于PIC16F877单片机的智能垃圾桶","uri":"/posts/pic16f877-ljt/#"},{"categories":null,"collections":null,"content":"视频演示 ","date":"2015-11-21","objectID":"/posts/pic16f877-ljt/:0:0","tags":null,"title":"基于PIC16F877单片机的智能垃圾桶","uri":"/posts/pic16f877-ljt/#视频演示"},{"categories":null,"collections":null,"content":"汇编部分 status equ 03h portc equ 07h trisc equ 87h portd equ 08h ;hw,smoke trisd equ 88h porta equ 05h trisa equ 85h porte equ 09h ;re0蜂鸣器 低有效 trise equ 89h ;keyborad portb equ 06h trisb equ 86h ;=================定时器tmr0=================== tmr0 equ 01h option_reg equ 81h intcon equ 0bh tmr0b_fast equ 128 tmr0b_slow equ 0 ;==================常量定义=================== count equ 21h ;电机转数 ljt_status equ 22h ;桶盖状态，0为关闭，1为打开状态 jp_status equ 23h ;键盘状态 yw_status equ 24h ;yw ;===================体1设置========================= org 000h nop bsf status,5 bcf status,6 ;转到体1 movlw 00h ;电机四位信号输出 movwf trisc movlw 0ffh movwf trisd ;红外信号一低两位输入,0为遮挡 movlw 0ffh ;设置ra为输入 movwf trisa movlw 0ffh ;rb输入 movwf trisb movlw 00h ;蜂鸣器输出 movwf trise movlw 03h ;分频数 movwf option_reg bcf option_reg,3 ;分频器分配给tmr0 bcf option_reg,4 ;上升沿tmr0递增 bcf option_reg,5 ;内部时钟提供时钟源 bcf option_reg,7 ;启用b端口上拉电阻 goto init ;=====================体0设置===================== init bcf status,5 ;转到体0 movlw 00h ;输出初始化 movwf portc movwf ljt_status ;桶盖初始状态为0 movlw 0ffh ;蜂鸣器初始不工作 movwf porte movlw 01dh ;电机初始转数 movwf count nop nop goto main ;======================主程序==================== main call scan ;调用键盘扫描 call yw_gy movlw 01dh ;电机初始转数 movwf count btfsc portd,0 goto work1 btfss ljt_status,0 ;若桶盖为1,跳过正转 call djzz ;电机正转 work1 movlw 01dh ;电机初始转数 movwf count btfss portd,0 goto main btfsc ljt_status,0 ;若桶盖为0,跳过反转 call djfz ;电机反转 goto main ;==============键盘扫描=============== scan ;第一行扫描 movlw B'11111110' movwf portb nop nop bsf status,5 ;到体1，转换方向 movlw 0f0h movwf trisb bcf status,5 ;返回体0 movf portb,0 movwf jp_status ;键盘状态读入通用寄存器 btfss jp_status,4 goto set_flag1 btfss jp_status,4 goto scan btfss jp_status,5 goto set_flag2 ;第二行扫描 bsf status,5 ;到体1，转换方向 movlw 0fh movwf trisb bcf status,5 ;返回体0 movlw B'11111101' movwf portb nop nop bsf status,5 ;转换方向 movlw 0f0h movwf trisb bcf status,5 ;返回体0 movf portb,0 movwf jp_status btfss jp_status,4 goto set_flag3 btfss jp_status,5 goto set_flag4 return ;============标志测试============= set_flag1 ;电机正转打开桶盖 movlw 01dh ;电机初始转数 movwf count call djzz bsf ljt_status,0 return set_flag2 ;电机反转关闭桶盖进入工作状态 movlw 01dh ;电机初始转数 movwf count call djfz bcf ljt_status,0 return set_flag3 ;无 return set_flag4 ;无 return ;=====================电机正转===================== djzz call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast movlw 080h movwf portc call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast movlw 0c0h movwf portc call delay_fast call delay_fast call delay_fast call delay_fast movlw 040h movwf portc call delay_fast call delay_fast movlw 060h movwf portc call delay_fast call delay_fast movlw 020h movwf portc call delay_fast call delay_fast movlw 030h movwf portc call delay_fast call delay_fast movlw 010h movwf portc call delay_fast call delay_fast movlw 090h movwf portc call delay_fast call delay_fast decfsz count,1 goto djzz bsf ljt_status,0 ;桶盖打开标志1 return ;======================电机反转==================== djfz call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast movlw 090h movwf portc call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast call delay_fast movlw 010h movwf portc call delay_fast call delay_fast call delay_fast call delay_fast movlw 030h movwf portc call delay_fast call delay_fast movlw 020h movwf portc call delay_fast call delay_fast movlw 060h movwf portc call delay_fast call delay_fast movlw 040h movwf portc call delay_fast call delay_fast movlw 0c0h movwf portc call delay_fast call delay_fast movlw 080h movwf portc call delay_fast call delay_fast call hwgy_tg_open ;检测满，是则蜂鸣 decfsz count,1 goto djfz bcf ljt_status,0 ;桶盖关闭0 call delay_fast call hwgy_tg_close ;检测清空 return ;==============红外感应=================== ;红外感应使桶盖打开 rd0 hwgy_open call delay_slow call delay_slow call delay_slow btfsc portd,0 goto hwgy_open return ;红外感应使桶盖关闭 rd0 hwgy_close call delay_slo","date":"2015-11-21","objectID":"/posts/pic16f877-ljt/:0:0","tags":null,"title":"基于PIC16F877单片机的智能垃圾桶","uri":"/posts/pic16f877-ljt/#汇编部分"},{"categories":null,"collections":null,"content":"ingbyr ","date":"0001-01-01","objectID":"/about/:1:0","tags":null,"title":"关于我","uri":"/about/#ingbyr"}]