mai1015

Microk8s Install Cert-manager

2021-02-01T00:00:00+00:00

Install

First you will need to install microk8s and enable helm3 and ingress using microk8s enable helm3 ingress.

Then install cert-manager with helm with following cmd by (Envekus)[https://www.reddit.com/r/kubernetes/comments/g3z5sp/microk8s_with_certmanager_and_letsecncrypt/].

microk8s kubectl create namespace cert-manager
microk8s helm3 repo add jetstack https://charts.jetstack.io
microk8s helm3 repo update
microk8s helm3 install cert-manager jetstack/cert-manager \
  --namespace cert-manager --version v0.15.2 \
  --set installCRDs=true \
  --set ingressShim.defaultIssuerName=letsencrypt-production \
  --set ingressShim.defaultIssuerKind=ClusterIssuer \
  --set ingressShim.defaultIssuerGroup=cert-manager.io

Then setup cluster issuer with the following cmd.

microk8s kubectl apply -f - <<YAML
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    email: CHANGE-ME@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-production-issuer-account-key
    solvers:
    - selector: {}
      http01:
        ingress:
          class: public
YAML

Then you will be able to gerenate the certificate. The ingress class in microk8s is public for me. Otherwise, it is not generating endpoint for ACME validation.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: example
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "public"
    cert-manager.io/cluster-issuer: "letsencrypt-production"
spec:
  tls:
    - hosts:
        - example.com
      secretName: example-tls-acme
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            backend:
              serviceName: example
              servicePort: 8080

Alipay Sdk Learn

2021-01-31T00:00:00+00:00

Istio Upstream Connection Overflow

2021-01-20T00:00:00+00:00

Backgroun

I found some failed request 503 error for the microservice deployment in Openshift 4.5 with Service Mesh v1. The error shown is upstream connect error or disconnect/reset before headers. reset reason: overflow.

Problem

I found the solution during the search and refer to the documentation. It is because Service Mesh v1 is using (Istio 1.3)[https://istio.io/v1.3/docs/reference/config/networking/v1alpha3/destination-rule/#ConnectionPoolSettings-HTTPSettings]. And the Connection poll is set to 1024 for http1MaxPendingRequests, http2MaxRequests, etc. It is different from version 1.8 where it is default to 2^32-1. Therefore, it is required to set it differently.

Solution

This is not really a good solution I believe. and it require sometime to investigate, but you can increase those settings with destination rules.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: sample-destinationrule
spec:
  host: sample
  trafficPolicy:
    loadBalancer:
      simple: RANDOM
    connectionPool:
      http:
        http1MaxPendingRequests: 10240
        http2MaxRequests: 10240
  subsets:
  - name: v1
    labels:
      version: "v1"

Kubernetes Debugging Network

2020-12-15T00:00:00+00:00

When you try to debug the network inside kubernetes but not much tool avaliable since all the image remove unused command. The easy way to to use busybox by deploying a pod using the following commandline kubectl run -i --tty --rm debug --image=busybox --restart=Never -- sh. Then you can access to telnet, nc, etc.

Kube Rolling Update

2020-11-15T00:00:00+00:00

https://stackoverflow.com/questions/53591417/kubernetes-kubectl-apply-does-not-update-pods-when-using-latest-tag

Using ESLint and Prettier with React

2020-07-20T00:00:00+00:00

How to setup ESLint and Prettier for your React apps Using ESLint and Prettier in a TypeScript Project eslint-config-ionic How to use ESLint with TypeScript

RCF2047 mutipart form decode for php

2020-05-09T00:00:00+00:00

Sometime, there are error on the php backend when uploading the file. When I check the form data and it was encoded with RCF2047.

=?utf-8?B?L2RhdGEvdXNlci8wL2NvbS5mdXp6eXBhd3Muc25zL2ZpbGVzL+iMuOS5i+eIqsK356S+5Yy6L0ltYWdlcy8yMDIwMDUwODIyNTExMjMzODMuanBlZw==?=

This is because the file path contains non-US ACSII character, so the request programe will try to encode it first. There is no way to extract file name from the raw data. The easy way to solve it is using iconv_mime_decode function to decode the file name first.

echo iconv_mime_decode('=?utf-8?B?L2RhdGEvdXNlci8wL2NvbS5mdXp6eXBhd3Muc25zL2ZpbGVzL+iMuOS5i+eIqsK356S+5Yy6L0ltYWdlcy8yMDIwMDUwODIyNTExMjMzODMuanBlZw==?=', 0);

The filename will output correctly.

TS2345 error using react-redux connect

2020-05-05T00:00:00+00:00

TL;DR

install @type/react-redux@6.0.6

npm i --save-dev @type/react-redux@6.0.6

Then it will be ready to go.

What cause the problem

When trying to connect state with connect function shown belown.

export interface ILoginProps extends StateProps, DispatchProps {}
const Login: React.FC<ILoginProps> = (props) => { /** codes omitted **/ };

const mapStateToProps = ({ authentication }: IRootState) => ({
    isAuthenticated: authentication.isAuthenticated
});
const mapDispatchToProps = { loginUser };
type StateProps = ReturnType<typeof mapStateToProps>;
type DispatchProps = typeof mapDispatchToProps;

export default connect(
    mapStateToProps,
    mapDispatchToProps
)(Login);

The newer @type/react-redux@7.1.7 is likely to cause problem because it is trying to resolve the dispatch function into different form.

// from
const loginUser = (username:string, password:string, rememberMe:boolean) => async (dispatch, getState) => Promise<Void>
// to
const loginUser = (username:string, password:string, rememberMe:boolean) => Promise<Void>

This will cause this extend ILoginProps does not match with type expected with connect() function.

Solution

Therefore, the easy is to downgrade by install the package below. The other way is to manually write the prop interface.

export interface ILoginProps extends StateProps {
    loginUser: Function
}

After more search, I found this answer. The document also has more information. This is another way to solve the problem.

export interface ILoginProps extends PropsConnected {}
const mapStateToProps = ({ authentication }: IRootState) => ({
    isAuthenticated: authentication.isAuthenticated
});
const mapDispatchToProps = { loginUser };

const connector =  connect(
    mapStateToProps,
    mapDispatchToProps
);
type PropsConnected = ConnectedProps<typeof connector>;

export default connector(Login);

Spring Boot Autowiring Multiple implementation for Interface

2019-12-24T00:00:00+00:00

Autowiring is a convenient feature for development. Developers do not require to handle object creation, the application can help decide the object/s that a service or controller required to work.

When specifying a class, sometimes it is necessary to have an interface so you can make different implementation. However, it might confuse the application since it cannot choose the correct bean. There are ways to solve the problem.

Primary annotation.

If you have multiple implementations, you can choose the one that is primarily used. Then apply @Primary, the application will automate wire all related interface to this implementation.

public interface A {
	void a();
}

@Service
@Primary
public class B implements A {
	//...
}

@Service
public class C implements A {
	//...
}

So the application will link it to class B.

Qualifier annotation

If you need to choose between different implementation, you should use @Qualifier.

public interface A {
	void a();
}

@Service
@Qualifier("firstService")
public class B implements A {
	//...
}

@Service
@Qualifier("secondService")
public class C implements A {
	//...
}

If you need implementation C, you can specify with `@Qualifier`.

@Component
public static class FirstManager {

    private final A a;

    @Autowired // inject FirstServiceImpl
    public FirstManager(@Qualifier("secondService") A a) {
        this.a = a;
    }
}

Reference: link

Testing in Spring Boot with JPA

2019-12-23T00:00:00+00:00

Application needs to be tested. Here are some notes to test spring boot application.

Deploy kubernetes and helm (国内部署 kubernetes)

2019-08-13T00:00:00+00:00

国内部署 k8s 是真的难，坑太多了。这两个花了两天终于成功部署一个能用都了

尝试

首先，我从alpine系统下尝试部署，可是源里面都 test kubernetes 不能用。运行 kubeadm init 每次都是部署失败，即使kubelet启动成功了。但是卡在docker这里，因为其他组件都不会成功运行。日志也没有任何相关资料，github上都问题解决方案都没有用。所以，必须得换一个系统。并且国内墙了google，源都不能访问。

过程

我选用了Debian 10作为我都主系统。

Docker

安装完成后，可以直接通过 curl https://get.docker.com/ | bash - 来安装docker。不知道为什么 Debian 10 安装不上 cri-o。docker 用起来也没问题，所以就不在这里纠结了。然后检查一下运行状况，没问题，修改一下配置。

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "insecure-registries" : [
    "10.0.1.121:5000",
    "127.0.0.1:5000"
  ],
  "registry-mirrors": ["https://************.myhuaweicloud.com"],
  "storage-driver": "overlay2",
  "data-root": "/mnt/docker"
}

这里我配置了华为的镜像，需要先在这里注册，就能获取一个链接。kubeadm 推荐systemd 作为 cgroup 的驱动，所以改动一下。

Kubernetes

之后就可以开始安装 k8s了。参考了一下这些文章：

按照第一篇文章「木子甘」，先进行了系统的设置

echo -e "net.bridge.bridge-nf-call-ip6tables =1\nnet.bridge.bridge-nf-call-iptables =1\nnet.ipv4.ip_forward = 1" >> /etc/sysctl.conf;sysctl -p;swapoff -a;sed -ri "/swap/s@(.*)@#/&@g" /etc/fstab;echo "SELINUX=disabled" > /etc/selinux/config

然后加入阿里云的源来按照k8s的主要程序。

apt-get install -y apt-transport-https curl && \
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg |apt-key add -  && \
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main"  >>/etc/apt/sources.list.d/kubernetes.list  && \
apt update

之后就可以固定住他们的版本了，这样能保证稳定性。

apt-mark hold kubelet kubeadm kubectl docker-ce

在配置开始之前，先获取下docker的镜像：

IMAGE="kube-apiserver:v1.15.2 kube-controller-manager:v1.15.2 kube-scheduler:v1.15.2 kube-proxy:v1.15.2 pause:3.1 etcd:3.3.10 coredns:1.3.1"
for imageName in ${IMAGE} ; do
    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
    
    docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
done

这里从其他源下载了镜像，然后改成k8s的名字。注意版本要按照刚刚安装的版本一样，之后就可以开始配置k8s了。

然后就可以通过kubeadm进行初始化了。这里设置了内网的ip，还有api服务器的ip。

kubeadm init --pod-network-cidr=10.0.1.0/16 --apiserver-advertise-address=10.0.1.***

之后，如果没有报错k8s就算成功运行了。

配置账号

如果运行成功，他会提示你配置一下你的账号。要不然不能管理服务。按照提示输入就好了。

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

配置网络

在这k8s成功运行但是他还没准备好。因为网络策略还没配置。详细的配置程序在官方文档有很详细的说明。我挑选了calico。

kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml

至此，k8s就算准备好了。但是它默认禁止了主节点不会运行其他的程序，所以我们要先把那个去掉

kubectl get no -o yaml | grep taint -A 5
kubectl taint nodes --all node-role.kubernetes.io/master-

这样就大功告成了

Helm

参考了这位大大写的kunernets使用helm安装tiller踩坑。真的解决了我的问题。 Helm安装很简单，直接从github上的release会有各个版本的编译文件。直接下载下来放在/usr/local/bin就好。之后就可以进行配置了。首先，如果打开了k8s的rbac，那么我们得先建立一个账号。

apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system

写入到rbac-config.yaml就可以用命令导入账号了。

$ kubectl create -f rbac-config.yaml
serviceaccount "tiller" created
clusterrolebinding "tiller" created
$ helm init --service-account tiller --history-max 200

但是这里也是得先获得一个tiller的镜像。我在docker hub上面找到了doublemine/kubernetes-helm.tiller的镜像。然后可以参照这个文档来签名。然后带上一个helm的源就可以开始配置了。我们还是选择阿里云的源。

helm init --debug --tiller-tls --tiller-tls-cert ./tiller.cert.pem --tiller-tls-key ./tiller.key.pem --tiller-tls-verify --tls-ca-cert ca.cert.pem --tiller-image doublemine/kubernetes-helm.tiller:v2.14.3 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts --service-account tiller

这里安装就完成了。

其他可能有用的

Spilt traffic using route table in mac

2019-08-11T00:00:00+00:00

When working remotely, we need to access internal resource. Due to security concern, that they are often restricted to only access from local. Then, we need to connect to vpn that enable us to interact with internal server.

However, we do not want to send all the request over vpn, since it might be slow. How do we control how traffic over vpn?

This can be achive by using route command.

Connect to the vpn, uncheck send all traffic over VPN connection
Delete route to vpn by default. sudo route delete -net default -interface ppp0
Add route to physical interface. sudo route add -net 0.0.0.0 -interface en0
Add ip range route to vpn. sudo route add -net 10.0.1.0 -netmask 255.255.255.0 -interface ppp0

Then you are done, and only this ip with in the range will send via vpn. Source: link

Install Oracle Java from source

2019-06-20T00:00:00+00:00

Background

I am using jhipster to generate spring boot project. It compile but it does not work well. The first thing is that some function is not compatible. Also, hibernate insert data twice that cause ConstraintViolationException. I had post a Issue.

It works well on Java 8, so I have to install one more version. However, I cannot install via PPA. It is because webupd8 has DISCONTINUED since we need to login to download. So, I have to download java manually. I can use update-alternatives to switch version.

Install

Download 1.8 jdk from Oracle. Register an account and then download. It is better to download jdk-8u211-linux-x64.tar.gz.
scp upload the file to server.
unzip the file at server with tar xzvf filename.
Move the folder into /usr/lib/jvm/ (The location my java 11 was installed).

Then use update-alternatives with commands.

update-alternatives --install /usr/bin/java java /usr/lib/jvm/java-8-oracle-amd64/bin/java 100
update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/java-8-oracle-amd64/bin/javac 100
update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/java-8-oracle-amd64/bin/jar 100

Update command to specify version.

update-alternatives --config java
update-alternatives --config javac
update-alternatives --config jar

Check java version if it works.
write $JAVA_HOME to specify location. I write it to /usr/lib/jvm/default-java and link to directory with ln.

Then when you switch version, you just change the link to specify version. Inspired by Rahul.

cUrl Test in local

2019-06-12T00:00:00+00:00

So, when we try to access our local development website, we tend to access the website via http://localhost:8080/. However, when we need to test for domain name functionality, we need to manually resolve it in hosts.

With cURL, we can easily resolve the domain manually. Use --resolve HOST:PORT:ADDRESS Force resolve of HOST:PORT to ADDRESS, we can change the address of the host.

# access http
curl --resolve 'mai1015.com:80:127.0.0.1' http://mai1015.com/path
# access http
curl --resolve 'mai1015.com:443:127.0.0.1' https://mai1015.com/path

So, the port is included in part of the resolve.

Or we can use -H, --header LINE Pass custom header LINE to server (H) to hack the header. We can change the host in the header. And, the server assumes it is accessing the specified server.

curl -H 'Host: mai1015.com' http://127.0.0.1/something

However, it does not work in https because the certificate domain does not match the URL. It is better to use --resolve.

Source: stackoverflow

Javascript open new Window blocked when asynchronous

2019-05-23T00:00:00+00:00

Problem

When you trying to open a new tag for your user. It is ok if we just use window.open right away. However, we need to do complex task like making request from server to generate the link. Then it will be block by the browser.

Solution

The solution is pretty straight forward. You can open new tab and modify it after you get the link. This is the sample code below.

async clicked(e) {
    // create new window
    var w = window.open('', '_blank');
    // fetch url
    var url = await someFunction();
    // then modify the location
    w.location.href = w;
}

The reason of that is the tab should be open during that user click event loop. Otherwise, it is going to assume that is a pop-up that should be blocked. That is why you can get ad pop-up on some site when you click.

This is from stackoverflow

Vuex Error With TypeScript

2019-05-18T00:00:00+00:00

Vuex is a cool library for vue that allow the project to control state easier. However, I was trying to use it with typescript. It causes error like this. I use championswimmer/vuex-module-decorators library to support vuex annotation.

ERR_ACTION_ACCESS_UNDEFINED: Are you trying to access this.someMutation() or this.someGetter inside an @action?
That works only in dynamic modules.
If not dynamic use this.context.commit("mutationName", payload) and this.context.getters["getterName"]
Error: Could not perform action login
at Store.eval (webpack-internal:///./node_modules/vuex-module-decorators/dist/esm/index.js:311:33)
at step (webpack-internal:///./node_modules/vuex-module-decorators/dist/esm/index.js:96:23)
at Object.eval [as throw] (webpack-internal:///./node_modules/vuex-module-decorators/dist/esm/index.js:77:53)
at rejected (webpack-internal:///./node_modules/vuex-module-decorators/dist/esm/index.js:68:65)
undefined

I dont understand why it happends because I did not acess any mutation or getter. I was confused. So, I modify my code again and again to see whats wrong. I return a Promise in action and I rejected it. And it cause the same error. So, I search it on google. It turns out that is an issue. The library warps the error, so the error is confusing. All I did then is add rawError option to the annatation so it becomes @Action({rawError: true}). And it display error normally.

Useful Git commands and Clone local

2019-05-18T00:00:00+00:00

Git is a strong version controll system. It allows you to track code and verison, and keep all the data you need to go back to certain version.

This is so good that we can use it even if we are working solo. We can see how the project grows. So, this is how to setup a project locally.

Steps

Here are some steps to prepare you own repository.

you will need to create a project by cd /path/to/project/. Then use git init to init a new git repository.
Then you will need to add your project files. git add .
Finally git commit -m "Inital" to commit you first changes.

So, those step setup the basics. As you go futher, you will get more files in your project. Remember to add and commit it as you go.

Adding file to git: git add file
Commit: git commit -m "message"

If you only change some file, you can use git commit -am "message" to make a fast commit. It saves all your time to add files that changes.

If you want to make it easier to share, you can init a bare repository git init --bare /path. This does not really contain your work file as usual. You have to clone and commit it if you want to modify the file.

Clone

If we want to get codes from other, we can use git clone git://... to clone other repository. and we can even clone our repository like git clone /path/to/project/ /path.

So, you might say, why would you do it. It is because you want to keep an deployment copy else where that does not change a lot.

Update

You got your copy of code, but you want to keep it up to date if you working with others. The command is git pull.

Simple Deployment

So you are done with you coding. Now, you want to deploy you code, you can try this gist. Add hooks that runs every commit.

Nginx reverse proxy solve no avaliable

2019-05-14T00:00:00+00:00

Background

We use a main server reverse proxy all the request to other backend server. So, it means that the client access the main server, and nginx reverse proxy the request to the other serever. So, we use subdomain to dynamically resolve the address so it is working well when server ip changes.

Problem

Today, the nginx is not working as expected. It redirect all request to 502 page. And we found the error log.

2019/05/14 08:41:15 [error] 688#688: *4 connect() failed (110: Connection timed out) while connecting to upstream, client: 113.*.*.*, server: furry.top, request: "GET / HTTP/2.0", upstream: "https://116.*.*.*:443/", host: "furry.top", referrer: "https://furry.top/"
2019/05/14 08:41:15 [error] 688#688: *4 no live upstreams while connecting to upstream, client: 113.*.*.*, server: furry.top, request: "GET / HTTP/2.0", upstream: "https://https/", host: "furry.top", referrer: "https://furry.top/"

It is quite weird because when we reload the server it starts working. And then stop working after 10 seconds. We could not found why this happend.

Attempts

I assume that it is because of the unstable connection. So, I check append max_fails=0 config to ensure upstream does not fail. However, it gets worse. Nginx basically try to connect to backend continuously and not reply to any connection. However, when I curl backend address, it has successfully responsed. So nothing wrong with the connection but nginx some how cannot connect to the backend.

I have no idea why it happend. Then I check the connection with netstat -ap. One line draw my attention.

Proto Recv-Q Send-Q Local Address        Foreign Address   State     PID/Program name
tcp        0      1 main-server.a:36298  166.*.*.*:http    SYN_SENT  10128/nginx: worker

Since there is no respond from the address, it did not connect to correct backend address. But, why it only works for small period of time? Does it mean that the dns server did not work?

So I check in the configuration file again. I found resolver 114.114.114.114, this changes the DNS server. So, to verify the address. I use dig command, dig @114.114.114.114 furry.top and the result is that 114.114.114.114 is not working. So, I replace the DNS with 8.8.8.8 and the problem solved.

So, that’s why nginx fail to connect to backend but I cannot find any log indicate that the DNS failed. It is because I did not check the default logging file.

[error] 2439#2439: upstream-dynamic-servers: 'furry.top' could not be resolved (110: Operation timed out)

The answer was there.

Keycloak Docker setup and reverse proxy from nginx

2019-05-05T00:00:00+00:00

Keycloak is an open source Identity and Access Management software that is part of Red Hat project. It provided OAuth and SSO support for your application and software. It is easy to set up, but you need to download the dependency and set up in the configuration file. We can use Docker to set this up easily.

From github repository, we can find the information about Keycloak for Docker. One of the simple setups is docker run -e KEYCLOAK_USER=<USERNAME> -e KEYCLOAK_PASSWORD=<PASSWORD> jboss/keycloak. Then it is going up and running. We can have a database connection when we set DB_VENDOR environment variable. There is a sample setup for Keycloak with MySQL. It is pretty easy with docker.

However, I do not need multiple MySQL instances running. I prefer a single MySQL service at my localhost. So, we need to connect host MySQL within the container. We have to locate the host IP address from the container. Docker used to provide host.docker.internal to resolve the host IP address. However, it resolves to nothing in Ubuntu docker. I found a simple workaround from here.

SETUP

Basically, it set the network mode to host, so localhost can access the actual machine. so I set it with docker-compose with network_mode: host. So the sample compose setup will look like this. With the file, we can set the database clear. The file is generated with jhipster, and I modify it to connect it to the database.

version: '2'
services:
  keycloak:
    image: jboss/keycloak:5.0.0
    network_mode: host
    command: ["-b", "0.0.0.0", "-Dkeycloak.migration.action=import", "-Dkeycloak.migration.provider=dir", "-Dkeycloak.migration.dir=/opt/jboss/keycloak/realm-config", "-Dkeycloak.migration.strategy=OVERWRITE_EXISTING", "-Djboss.socket.binding.port-offset=1000"]
    volumes:
      - ./realm-config:/opt/jboss/keycloak/realm-config
    environment:
      - PROXY_ADDRESS_FORWARDING=true
      - KEYCLOAK_USER=admin
      - KEYCLOAK_PASSWORD=admin
      - DB_VENDOR=mariadb
      - DB_DATABASE=keycloak
      - DB_USER=keycloak
      - DB_PASSWORD=passwd
    ports:
      - 9080:9080
      - 9443:9443
      - 10990:10990

It is important to add PROXY_ADDRESS_FORWARDING=true since we want to working with reverse proxy nginx.

We can then run the file with docker-compose -f keycloak.yml up. However, it does not work because of the message Failed to start service org.wildfly.network.interface.private. There is nothing I can find on the internet, but it works fine on my other computer. It should be a problem with docker in Ubuntu. So, I have used another way. Please comment if you anything related to the issue.

MYSQL

We have to know the IP address of the host in the container. So, I found this command ip route show | awk '/default/ {print $3}'. It will show the host IP in the docker. We can simply do it like docker run --rm alpine ip route show | awk '/default/ {print $3}', and we will see the IP in terminal. For me, the IP is 172.17.0.1. So I add DB_ADDR=172.17.0.1 into the environment. And let myself listen to IP address 0.0.0.0 then the container is allowed to connect to the service.

One more thing is the MySQL user. We can add a user with remote access, but it is not safe. So I add the user in MySQL with SQL command.

CREATE USER 'keycloak'@'172.*' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON `keycloak`.* TO 'keycloak'@'172.*';
FLUSH PRIVILEGES;

or we can simply do GRANT ALL PRIVILEGES ON keycloak.* To 'keycloak'@'172.*' IDENTIFIED BY 'password';. They both work well with mariadb-10.3.14.

NGINX

We want to proxy it with nginx so we do not need to convert certificate. It is easier to replace and do not stop the keycloak service. So, I obtain a certificate with certbot and activate it with Nginx configuration like this.

nginx host configuration

server {
    listen 80;
    listen [::]:80;
    server_name example.com;

    location / {
        rewrite ^ https://example.com$request_uri? permanent;
    }
}

server {
    server_name example.com; # managed by Certbot

    access_log /var/log/nginx/ide.mai1015.com-access.log;
    error_log /var/log/nginx/ide.mai1015.com-error.log;

    # SSL configuration

    listen 443 ssl http2;
    # listen [::]:443 ssl http2 ipv6only=on;

    # Note: You should disable gzip for SSL traffic.
    # See: https://bugs.debian.org/773332
    #
    # Read up on ssl_ciphers to ensure a secure configuration.
    # See: https://bugs.debian.org/765782
    #
    # Self signed certs generated by the ssl-cert package
    # Don't use them in a production server!
    #
    # include snippets/snakeoil.conf;

    # Add index.php to the list if you are using PHP
    index index.html index.htm index.nginx-debian.html;

    location / {
        proxy_pass http://localhost:9080;
        proxy_read_timeout 90;

        # proxy header
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Scheme $scheme;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $host;

        # ws
        proxy_http_version 1.1;
        #proxy_set_header Upgrade $http_upgrade;
        #proxy_set_header Connection 'Upgrade';

        # proxy_cache_bypass $http_upgrade;

        #proxy_redirect http://localhost:9080 https://example.com;
    }

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot
    # include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    # ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

So those configurations are important to allow keycloak to work correctly. Without Host header will break the redirect_url.

    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Host $host;

So we have a fully configurated keycloak working behind reverse proxy Nginx.

Ansible Automation for Server Deploy and Test

2019-05-01T00:00:00+00:00

When talking about deploying a server, it is a pain to configure servers by hand. You have to type in commands line by line. Then edit the config file to make sure that works. If you need to configure all the time, it wastes a lot of time struggle and waits for server.

Then I look for a way to automate this process. I found a lot of program like chef, puppet, ansible, saltstack, terraform, cloudformation, etc. They are different but they serve one purpose, deploy servers for you.

First, I tried chef. It is cool, clear syntax, fast. I can always check if the setup was correct by config rules. However, after some day of experience, I give up. It has so many stuff to use, I have to keep search and try it out. If I am managing a lot of servers, I will definitely use it.

Then I found ansible. It is pretty easy to learn, the configuration file is using YMAL. One of the samples from documents.

---
- hosts: webservers
  vars:
    http_port: 80
    max_clients: 200
  remote_user: root
  tasks:
  - name: ensure apache is at the latest version
    yum:
      name: httpd
      state: latest
  - name: write the apache config file
    template:
      src: /srv/httpd.j2
      dest: /etc/httpd.conf
    notify:
    - restart apache
  - name: ensure apache is running
    service:
      name: httpd
      state: started
  handlers:
    - name: restart apache
      service:
        name: httpd
        state: restarted

It is easy to both read and writes. It enables you to config server via ssh using ansible in your computer.

I learn a few, so I will take some example and show how ansible works and automate the process.

After the installation guide, you will configure the host that you want to config in groups. We create the file /etc/ansible/hosts to declare individual servers or server group.

192.0.2.50

[group]
aserver.example.org
bserver.example.org

Or if you want to move the file location, you can create ~/.ansible.cfg and put in the following content. It specifies the location of the host file.

[defaults]
inventory = /path/to/hosts

And we can start with a playbook. A playbook is the configuration file that contains what servers to deploy and the tasks to do. We can start with nginx installation.

---

- name: ide source
  hosts: local
  tasks:
  - name: install nginx
    apt:
      name: 'nginx'
      update_cache: yes
      state: latest

This file will try to config all the server under local group and then use apt to install nginx to latest version after updated the cache. It is pretty clear. Or we can install multiple packages.

- name: Install common
  become: yes
  apt:
    name: ['git', 'gdb', 'build-essential']
    update_cache: yes
    install_recommends: no

After that, we can make sure that nginx is started.

- name: restart NGINX
  service:
    name: nginx
    state: started

So, that’s all the basic function that ansible able to do easily. It also works with loop, I can install a list of item of go with. I have to add environment variable so I just activate the .profile.

- name: install go ext
  become_user: ""
  shell: ". /.profile && go get -u "
  with_items:
    - github.com/stamblerre/gocode
    - github.com/uudashr/gopkgs/cmd/gopkgs
    - github.com/ramya-rao-a/go-outline
    - github.com/acroca/go-symbols
    - golang.org/x/tools/cmd/guru
    - golang.org/x/tools/cmd/gorename
    - github.com/rogpeppe/godef
    - github.com/zmb3/gogetdoc
    - github.com/sqs/goreturns
    - golang.org/x/tools/cmd/goimports
    - golang.org/x/lint/golint
    - github.com/alecthomas/gometalinter
    - honnef.co/go/tools/...
    - github.com/golangci/golangci-lint/cmd/golangci-lint
    - github.com/mgechev/revive
    - golang.org/x/tools/cmd/gopls
    - github.com/go-delve/delve/cmd/dlv

They are all useful and easy to maintain config.

And it provides role that allow reusable playbook. Then I just need to write

- name: ide source
  hosts: ideserv
  become: yes
  roles:
    - ide
    - web

Then run ansible-playbook coder.yml and it will try to log in and do all the stuff for me.

Windows Subsystem Linux Development

2019-04-19T00:00:00+00:00

Linux SHELL is very useful in development. It is color coded, easier to manage, and more command. Windows provide WSL to enable linux system in windows. To enable it this is the official guide.

Basically, you need to enable Windows Subsystem for Linux in the program windows fratures. Then select one of the linux system in Microsoft Store. The one I recommand is Debian. It is small and Ubuntu is base on Debian.

After you restart and run Debian, linux will be ready to go. You now running full operational linux in windows. And it can access windows directories.

Installation will be easier. For git, just run sudo apt install git.

I try to let IDE (Intellij IDEA) to run git through wsl, I found a bat file from here.

@echo off
setlocal enabledelayedexpansion
set command=%*
set find=C:\Users\%USERNAME%\AppData\Local\Temp\git-commit-msg-.txt
set replace=/mnt/c/Users/%USERNAME%/AppData/Local/Temp/git-commit-msg-.txt
call set command=%%command:!find!=!replace!%%
echo | C:\Windows\Sysnative\bash.exe -c 'git %command%'

It works beautifully.

This is far from a good development. Since there are more to install. And I install java in windows, so I can use it eaiser without running wsl java all the time.

Centos install newer package of gcc via yum

2019-04-09T00:00:00+00:00

Centos is popular for it is stability and security. It meets all kinds of purpose for commercial requirements. However, to maintain security and stability, most of its package are outdated. It is harder to run newer version compiled program with lower version of gcc.

We can solve this easily by compiler new version of gcc. It is not that hard, but requires some time to do the job. We can fix it in a easy way. Thanks to Software Colletions, it is provide in RHSCL(Red Hat Software Collections) in devtoolset-7. We just simply need to enable it and download it from the repository. It can be done in following steps. You can find more information here

Install the centos-release-scl in centos repository
```
sudo yum install centos-release-scl
```

Enable RHSCL repository

sudo yum-config-manager --enable rhel-server-rhscl-7-rpms

Install the collections
```
sudo yum install devtoolset-7
```
Enable the package
```
scl enable devtoolset-7 bash
```

You can find more other package from the website. I can also download node from here. So we do not need to spend time compile the software.

Scrapy grap files from website

2019-04-05T00:00:00+00:00

I am trying to download some material from my course website. However, there are a lot of files in the server. I do not want to download file one by one. So, I try to write a script that can help me download all the files at once.

I found scrapy. It is a crawler framework that can extract data from website.

First of all, install scrapy from source. In my case, I use conda instead of pip. It is simlar fo pip but I can setup different envirnment with conda. Installtion command is conda install -c conda-forge scrapy.

After install we can follow this guide. First, I create the project with scrapy startproject yorku that creates an sample configurated scrapy project. It looks like

scrapy/
   ├── yorku/
   │    ├── __init__.py
   │    ├── items.py
   │    ├── middlewares.py
   │    ├── pipelines.py
   │    ├── settings.py
   │    └── spiders/
   │        └── __init__.py
   └── scrapy.cfg

It contains only basic file for setting up spider, we can generate spider with command scrapy genspider <name> <domain name>. I typed in scrapy genspider wikieecs wiki.eecs.yorku.ca that generates the spider that starts like this.

import scrapy
import re
import os
from scrapy.utils.python import to_native_str

class WikieecsSpider(scrapy.Spider):

    name = 'wikieecs'
    allowed_domains = ['wiki.eecs.yorku.ca']
    start_urls = ['http://wiki.eecs.yorku.ca/']

    def parse(self, response):
        pass

We can write logic to seek the links to the file. So, it will start access the starts_urls. When it finish loading the page, it will pass the page in the response object to parse function. So, we can extract the link of the files from the page. It can be done easily with build in css seceltor or xpath.

response.css('a::attr(href)') # select all the link with url.

It returns an array of object. Then we can use loop to see all the links and select the files.

for url in response.css('a::attr(href)'):
    link = url.extract() # get the link from element
    if link.endswith(".pdf"): # if it is an pdf
        yield response.follow(link, self.save_file) # instrcut scrapy to download the file

So this function will generate a new array that conatains all the function. If you want to download multiple file with different ext. i.e. docx, pptx. You can type all stuff in to an array like this.

ext = ['.pdf', '.ppt', '.pptx', '.doc', '.docx', '.txt', '.v', '.c', '.tar', '.tar.gz', '.zip']
link.endswith(tuple(ext)) # if it ends with any of the ext

So you can download specified list of files. Then we need to generate a new callback save_file to save the file.

def save_file(self, response):
    filename = response.url.split("/")[-1] # assume the last path of url is the name
    with open(filename, 'wb') as f: # open a new file
        f.write(response.body)      # write content downloaded
    self.logger.info('Save file %s', filename) # display the file downloaded

However, our school website require us to login in order to download. And it uses basic access authentication. So we need to generate a new header Authorization if it require us to login. Scrapy support this with HttpAuthMiddleware by defaut, we just need to spcify this with http_user and http_pass in the spider.

# improt omitted
class WikieecsSpider(scrapy.Spider):
    http_user = 'xxxx'
    http_pass = 'xxxx'

    # other definition and functions

Then you will have a working spider. The final code will look like.

import scrapy
import re
import os
from scrapy.utils.python import to_native_str

class WikieecsSpider(scrapy.Spider):
    http_user = 'xxxx'
    http_pass = 'xxxx'

    name = 'wikieecs'
    allowed_domains = ['wiki.eecs.yorku.ca', 'www.eecs.yorku.ca']
    start_urls = ['https://wiki.eecs.yorku.ca/course_archive/2018-19/W/2011/assignments:start']
    
    ext = ['.pdf', '.ppt', '.pptx', '.doc', '.docx', '.txt', '.v', '.c', '.tar', '.tar.gz', '.zip']

    def parse(self, response):
        for url in response.css('a::attr(href)'):
            link = url.extract()

            if link.endswith(tuple(self.ext)):
                yield response.follow(link, self.save_file)
    
    def save_file(self, response):
        filename = response.url.split("/")[-1]
        with open(filename, 'wb') as f:
            f.write(response.body)
        self.logger.info('Save file %s', filename)

We just need put all the url in to the start_urls and it will start to scan the files and try to download it.

Recurrent Neural Networks

2019-03-15T00:00:00+00:00

\[\begin{align*} \frac{\partial\varepsilon}{\partial\theta}&=\sum_{1 \leq t \leq T} \frac{\partial\varepsilon_{t}}{\partial\theta}\\ \frac{\partial\varepsilon}{\partial\theta}&=\sum_{1 \leq k \leq t} (\frac{\partial\varepsilon_{t}}{\partial x_{t}}\frac{\partial x_{t}}{\partial x_{k}}\frac{\partial^{+}x_{k}}{\partial \theta})\\ \frac{\partial x_{t}}{\partial x_{k}}&=\amalg_{t \geq i > k} \frac{\partial x_{i}}{\partial x_{i-1}}=\amalg_{t \geq i > k} W_{rec}^{T} diag(\sigma^{'}(x_{i-1})) \end{align*}\]

$W_{rec} \sim small (<1)$ cause vanishing
$W_{rec} \sim large (>1)$ cause Exploding

Solution to The Vanishing Gradient Problem.

Exploding Gradient
- Trucated Backpropagation
- Penalties
- Gradient Clipping
Vanishing Gradient
- Weight Initialization
- Echo State Networks
- Long Short-Term Memory Networks(LSTMs)

LSTM

Set $W_{rec}$ to 1. This is a good post about it. It basically saying that the tradictional neural network is hard to link between the context. However, they are good at linking it long-term. LSTMs is a way to solve it. It basically adds more action before the activation function, and that allows the

GTX1070 cannot work with Tensorflow

2019-03-15T00:00:00+00:00

I was trying to set up Deep learning on my Windows PC. However, after installing Anaconda, I cannot use my GPU to do my training. It shows

E tensorflow/stream_executor/cuda/cuda_blas.cc:461] failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED

I search for many sources trying to solve this problem. I tried to set up with this guide, and then I found this one with detailed. The setup was smooth, but I still cannot run the training because of the error.

It is definitely a nightmare to solve the issue. Then I found it on stackoverflow.

The solution is

add to the top of your code under import tensorflow as tf
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.Session(config=config...)

It works well when I modify the config “allow_growth” to true. Then it can use GPU to train the networked lighting fast.

UPDATE 2019-04-05: It was working well for some time. However, I found some serious problem with the graphic card. My GTX1070 crash everything if the memory usage exceed 3 GB. I found it when I was playing BF5. Now, I have sent the graphic card to repaired. So, if you run into some problem like this, you should try to check if your graphic card is working in a good condition.

C Pointer

2019-02-14T00:00:00+00:00

C pointer is powerful that can point to anywhere with good definition. Here are collections of interesting way to use pointer.

definition	meaning	link
`type *i;`	simple pointer	here
`type *i[n];`	array of n pointer	here
`type (*p)[n];`	a pointer to an array of n	here
`type (*p)();`	a pointer to function that return type	here
`type (*p[n])();`	a pointer to an array of n functions that return type	here

`type *i`

int j;
int *i;
i = &j;
*i = 10;
// j == 10

`type *i[n]`

int j;
int *i[10];
i[0] = &j;
*i[0] = 10;
// j == 10

`type (*p)[n]`

int i[10];
int (*p)[10];
p = &i;
*p[0] = 10;
*((*p)+1) = 11;
// i = {10,11,?,...,?}

`type (*p)()`

int test() {
	printf("%s\n", "test");
	return 1;
}
int main() {
	int (*x)();
	x = &test;
	int z = x();
	printf("%d\n", z);
}
//result:
//test
//1

`type (*x[n])()`

int test() {
	printf("%s\n", "test");
	return 1;
}
int main() {
	int (*x[10])();
	x[0] = &test;
	int z = x[0]();
	printf("%d\n", z);
}
//result:
//test
//1

Social Engineering Phishing

2019-02-06T00:00:00+00:00

This page is intended to show how I do during a social engineering phishing in EECS3482 at York University. This is not going to record any of the data you put in the form.

The process is simple, just download the whole page and modify the form. and it is done. It requires some HTML skill to understand how it works.

This is kind of attack that trick user put their password into a fake login form. And you will see your password in the URL bar in this case. That way, the attacker will record the password and find a chance to steal your account.

There are many ways to prevent. Simply look closely at the URL. Does it look different or is it SSL protected with a green lock and the name of the company.

Arduino Intergrated Circuit Communication Note

2019-02-05T00:00:00+00:00

I am trying to build a step counter with MPU6050, and it is not so hard to do it. I learn a lot of stuff relative to how hardware like arduino and other parts communicating.

Type of communication

I found three common type of communication protocals are widely used:

I2C (Inter-Integrated Circuit Protocol)
UART (Universal Asynchronous Receiver/Transmitter)
SPI (Serial Peripheral Interface)

I2C (Inter-Integrated Circuit)

This protocal is simple to use. It connects all part via I2C Bus which consist two lines are Serial Data Line (SDA) and Serial Clock Line (SCL).

A microcontroller act as master and other part act as slave takes controll from the master. The clock is controll by the master. Sameple schematic below, from wikipedia.

Each device have a unique address so thry can communicate. Basic communication is using transfers of 8 bytes. Each I2C slave have 7 bytes address. The bit 0 is signal to read or write to the device.

Communication is initialized by master device. In normal state, both lines SCL and SDA are high. Then Start condition is drop the SDA line. After that, ther master will start sending the address. Then follow by one bit of operation. If bit was 0, it means writing to slave. If bit was 1, then it will read from the slave. Once all bits are finish, it follows the Stop condition. Then the other device on the bus can use the bus. The image show how the device communicate, image from here.

UART (Universal Asynchronous Receiver/Transmitter)

This is not really a kind of communication protocol like I2C. It is a standalone IC. It only use two wires (two Tx pin to Rx pin) to transmit data between devices. This transmit data asynchronously, because there is no clock needed. Instead, it adds start and stop bit to the packet (Image below from here). So, it know when to start reading and end reading. They read byte at a specify frequency aka baud rate. Baud rate is the speed of data express in bit pre second (bps).

It works simple, however, you cannot connect multiple slave or mutiple master systems. Also, we need to adjuest the BPS to match the other part.

It’s no compatible with the I2C.

SPI (Serial Peripheral Interface)

I haven research about this. I have little knowledge of it. Thats how arduino and computer communicate. I will do some more research about it.

Full page background with css3

2019-01-20T00:00:00+00:00

I was trying to make a page with a full image background. It was simple. I fit everything into one div with class container. And give the background to the body tag.

body {
    background-image: url(/image/bg.jpg);
    background-repeat: no-repeat;
    background-size: cover;
}

It works well. However, I tried to make it responsive to different devices. It leaves an ugly white space at the bottom. I can tell that it is because the body did not have a value of height. I have to force it to have the height of the screen. Then, the code becomes:

body {
    height: 100%;
    background-image: url(/image/bg.jpg);
    background-repeat: no-repeat;
    background-size: cover;
}

It actually overdoes it. I now have a scroller at the right. This is so confusing for the unexpected result. After researching a little bit. The margin value actually pull the body down, so I got a margin for the body at the top.

There is a simple fix. I just add one more div that warps container, and give it overflow: auto;.

.page {
    overflow: auto;
}

Define New Conversions for printf

2019-01-07T00:00:00+00:00

In GNU C Library, there are ways to Customizing printf.

It allows you to create conversion specifiers. It allow you to print important data structures of our programe.

SSH refuse on bad ownership or modes

2019-01-05T00:00:00+00:00

After I updated the gitlab to 11.6.1-ee, ssh to git user failed.

I went on gitlab and did not found any useful information. Then I start checking my sshd services.

Jan  6 06:48:26 UltGeek-GitLabServer sshd[115940]: error: Unsafe AuthorizedKeysCommand "/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell-authorized-keys-check": bad ownership or modes for directory /opt/gitlab/embedded/service/gitlab-shell/bin
Jan  6 06:50:15 UltGeek-GitLabServer sshd[116182]: error: Unsafe AuthorizedKeysCommand "/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell-authorized-keys-check": bad ownership or modes for directory /opt

From the man page

AuthorizedKeysCommand
Specifies a program to be used to look up the user's public keys.  The program must be owned by root, not writable by group or others and specified by an absolute path.  Arguments to AuthorizedKeysCommand accept the tokens described in the TOKENS section. If no arguments are specified then the username of the target user is used.
The program should produce on standard output zero or more lines of authorized_keys output (see AUTHORIZED_KEYS in sshd(8)). If a key supplied by AuthorizedKeysCommand does not successfully authenticate and authorize the user then public key authentication continues using the usual AuthorizedKeysFile files.  By default, no AuthorizedKeysCommand is run.

It turns out to be the wrong onwership or modes for the directory after I updated it. Then I update all the relative directory including /opt because ssh panic when any of the directory is writable for other group or user with chmod 0755 dir.

Finally, it is back to working agagin. I do not sure if this makes any sense but it indeed get a better security.

Drop Inactive SSH Sessions

2019-01-03T00:00:00+00:00

There will be many connect remain if we unexpectedly disconnected from the server. It remains inactive but still shows up when you type w command.

How to drop an ssh connection when unexpectedly disconnect from the server?

we can command to solve the problem

ps -ef | grep sshd | grep -v root | grep -v 12345 | grep -v grep | awk '{print "sudo kill -9", $2}' |sh

ps -of displays all the process. Then we want to display only sshd session and drop your own ssh session. and then kill it with sh command.

just to remember to replace 12345 with your pid. It won’t kill any root user session.

If you are in linux you can try pkill -o -u YOURUSERNAME sshd to kill oldest SSH session.

Resource

Convolutional Neural Network

2018-12-29T00:00:00+00:00

Concept

Basic neural network is pretty simple way to build and let computer learn from stuffs. It can actually become pretty smart after some time. However, it is hard to let it work with things like images.

We have Convolutional Neural Network to make it easier.

Convolutional Neural Network

The basic concept are the same, but there are pre-processing for the image data.

1. Convolutional Operation

It takes the input image and apply Feature Detector via multiply each valuable value so respect to value to genertate Feature Map.

We will create multiple feature map with different Feature Detector to obtain first convolutional layer.

1b. ReLU Layer

Then we apply Rectifier Function to the Feature Map to break the linear because the image is not linear. Detail: Delving Deep into Rectifiers

2. Max Pooling

We then reduce the size of the Feature by doing a Max Pooling to generate a small Pooled Feature Map. It reducing the number of parameters going in to the network. Also, it is harder to get overfitting.

3. Flattening

It basically take the numbers row by row and put it in to long column. Then it can fit in to neural network for processing.

4. Full Connection

Rether than regular neural network, we have a fully connection network. It means the node is connected to all other nodes perivous layer.

Softmax & Cross-Entropy

Softmax

The result does not actually add up to one. so we apply a soft max to calculate the prectage.

\[\begin{align*} f_{i}(z)=\frac{e^{z_{j}}}{\sum_{k}e^{z_{k}}} \end{align*}\]

Cross-Entropy

Loss function.

\[\begin{align*} L_{i}=-\log{\frac{e^{f_{yi}}}{\sum_{j}e^{f_{j}}}} \end{align*}\]

Repesentation, is easier to calculate.

\[\begin{align*} H(p,q)=-\sum p(x) \log q(x) \end{align*}\]

Reinforcement Learning

2018-12-24T00:00:00+00:00

Concept

s - state
a - action
R - reward
$\gamma$ - Discount

Let AI know what they can do, and just let them know the things they can do.

such as a maze, tell AI agent to move around. Give a reward (Reinforcement Learning) to the agent.

How does agent remember to go out the maze? It distinguishes the path by giving it a value. One example is increasing the value of the path. However, we cannot assign the same value to all the action we take. Then we will introduce the Bellman equation.

Bellman Equation

$\begin{align*} V(s)=\underset{\scriptstyle\text{a}}{max}(R(s,a) + \gamma V(s')) \end{align*}$

s = current state
$s'$ = next state
R is reward while the game.

calculate the max reward, take the optimal state. $\gamma$ eliminate the same value in the path. Inspire the agent to go on the right path.

It basically builds a map.

Additional: Original Paper

Markov Decision Process

Markov Decision process is a mathematical framework that helps decide thing with some random and under control.

Using Bellman Equation, $V(s')$ becomes the expected value of all possible action. Does not know which state we are going into. $\begin{align*} V(s)=\underset{\scriptstyle\text{a}}{max}(R(s,a) + \gamma \sum_{s'} P(s, a, s')V(s')) \end{align*}$

p is the probability that the next step we take. It gives probability to a different movement to get close to the destination.

Markov Property

A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present states) depends only upon the present state, not on the sequence of events that preceded it. A process with this property is called a Markov process. (From wikipedia)

Interesting paper: A survey of applications of Markov decision processes

Living Penalty

When we have that calculation, we will have a better result. However, how do we encourage them to do it faster?

We can use punishment. For each movement, there will be a penalty. That way, it will try to go faster with less action.

Q-Learning Intuition

What is Q

Replace Q instead of V. $\begin{align*} V(s)&=\underset{\scriptstyle\text{a}}{max}(Q(s,a))\\ Q(s, a)&=R(s,a)+\gamma\sum_{s'} (P(s, a, s')V(s'))\\ Q(s, a)&=R(s,a)+\gamma\sum_{s'} (P(s, a, s')\underset{a'}{max}(Q(s', a'))) \end{align*}$

V takes the maximum value. Q leaning takes all the possible action.

In short: $Q(s, a)=R(s,a)+\gamma \:\underset{a'}{max}(Q(s', a')))$

Detail: Markov Decision Processes: Concepts and Algorithms

Temporal Difference

$\begin{align*} TD(a, s) = R(s,a)+\gamma\:\underset{a'}{max}(Q(s', a'))-Q(s,a) \end{align*}$

different in time. $Q(s, a)$ action previously, new $Q(s',a')$ now.

Want to change small amount $\begin{align*} TD(a, s) = R(s,a)+\gamma\:\underset{a'}{max}(Q(s', a'))-Q_{t-1}(s,a)\\ Q_{t}(s,a)=Q_{t-1}(s,a)+\alpha TD_{t}(a,s) \end{align*}$

$\alpha$ is the learning rate.

Get TD() as close as possible to 0. Come up the optimal action. $\begin{align*} Q_{t}(s,a)=Q_{t-1}(s,a)+\alpha (R(s,a)+\gamma\:\underset{a'}{max}(Q(s', a'))-Q_{t-1}(s,a)) \end{align*}$

Additional Reading: Learning to predict by the methods of temporal differences

Deep Q-Learning Intuition

Simple Q learning is no longer suited for the complicated environment. We can use deep learning to calculate Q. Therefore, it is not limited to one simple rule and formula.

The neural network will predict the Q value. How to adapt the TD in Q value. calculate error/loss using bp to update the weight.

Resource: Simple Reinforcement Learning with Tensorflow Part 4: Deep Q-Networks and Beyond

Experience Replay

In some environment, there is a different condition that will help AI to learn. Such as a self-driven car, it learns how to drive a car. Each movement is necessary to go through the network. However, it cannot learn will by one piece. Therefore, it will remember a sequence of action as a batch. When it has enough information, it started to learn by viewing the batch. It can enchant the learning process. It will reduce the number of time to practice.

Action Selection Policies

When the network output the percentage of an action. How does the machine choose which action to take? There are many action selection policies. One example is “softmax” decide the action we take.

In exploration, it will take as probability how often the action will take. Just randomly selecting the action.

Resource: Adaptive ε-greedy Exploration in Reinforcement Learning Based on Value Differences

how bit torrent works

2018-12-18T00:00:00+00:00

Bit torrent is currently pretty popular way to download, besides this, there is PT as well.

the advantage using bit torrent:

It is using peer-to-peer networking to do the file serving.
It contains the file information that checks if the file is corrupted.
It encourages people to upload their file, then people can download faster and share the portion that other client doesn’t have.
Torrent is open-sourced.

there will be a tracking server that keeps tracks all the people who downloaded the file, and they can make the file download faster when peer work together.

P2P concept study

2018-12-17T00:00:00+00:00

Concept

P2P is called peer-to-peer networking. It is kind of distributed application architecture.

Peers are equally privileged, equipotent participants in the application. Each peer is a node.

One example is the BitTorrent, there is a server as a tracker. It keeps all peers in one routing table. Each node will assign upon joining the network and assign a GUID. Then all peer that he contact will update the information and share to other peers.

Upon leaving the network, the node will contact other peers that he is going to leave. Peers are going to remove him from the routing table.

detail resources from: Peer-to-Peer (P2P) Networks

Choord vs. Kademlia

In Kademlia, routing table goes both ways. Peers have to notify others before leaving.

In Choord, they are not symmetric. They do not need to keep each other. when leaving, they have to send leave a message to the one keeps them in the routing table.

Introduction

2018-12-17T00:00:00+00:00

Hello, this is my new personal space to post blog and experiences.

I will update some note and tutorial from my learning.

RSS feed