nginx_site_gerrit

/etc/nginx/sites-available/gerrit

server {
    listen 80;
    server_name g.cf1g.com;
    return    301 https://$server_name$request_uri;
}

server {
    listen 443;
    server_name g.cf1g.com;

    location / {
        auth_basic              "Gerrit 2.12";
        auth_basic_user_file    /etc/nginx/htpwd.conf;
        proxy_pass              http://localhost:8081;
        proxy_set_header        X-Forwarded-For $remote_addr;
        proxy_set_header        Host $host;
    }

    ssl on;
    ssl_certificate 2016/1_cf1g.com_bundle.crt;
    ssl_certificate_key 2016/cf1g.rsa;

    ssl_session_timeout 5m;

    ssl_protocols SSLv3 TLSv1;
    ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
    ssl_prefer_server_ciphers on;
}

gerrit.config

[gerrit]
    basePath = git
    canonicalWebUrl = https://g.cf1g.com/
[database]
    type = postgresql
    hostname = localhost
    database = reviewdb
    username = gerrit2
[index]
    type = LUCENE
[auth]
    type = HTTP
[sendemail]
    smtpServer = localhost
[container]
    user = gerrit
    javaHome = /usr/lib/jvm/java-7-openjdk-amd64/jre
[sshd]
    listenAddress = *:29418
[httpd]
    listenUrl = proxy-https://127.0.0.1:8081/
[cache]
    directory = cache
[download]
  scheme = ssh
  scheme = http
  scheme = repo_download

Android boot image editor

This post is a clone of the README of my github repo Android_boot_image_editor
It’s targeted at those who need to modify existing Android boot.img in place, without downloading full AOSP code.


Utilies for editing Android boot.img.

Prerequisite

Host OS requirement:

Linux or Mac.

Target Android requirement:

(1) Targeted boot.img MUST follows AOSP verified boot flow, which means it packs linux kernel, rootfs , and a optional second state bootloader, then sign it with OEM/USER keys.

(2) These utilities are known to work for Nexus (or Nexus compatible) boot.img for the following Android releases:

  • Marshmallow (API Level 23)
  • Lollipop (API Level 21,22)
  • AOSP master

You can get a full Android version list here.

Usage

Get tools via git:

git clone https://github.com/cfig/Android_boot_image_editor.git
cd Android_boot_image_editor

Then put your boot.img at $(CURDIR)/boot.img, then start gradle ‘unpack’ task:

cp <original_boot_image> boot.img
./gradew unpack

Your get the flattened kernel and /root filesystem under $(CURDIR)/build/unzip_boot:

build/unzip_boot/
├── bootimg.json
├── kernel
├── second
└── root

Then you can edit the actual file contents, like rootfs or kernel.
Now, pack the boot.img again

./gradew pack

You get the repacked boot.img at $(CURDIR):

boot.img.signed

example & test

An example boot.img has been placed at src/test/resources/boot.img, which is extracted from Nexus 5x(code: bullhead) factory images from Google, you can take it as a quick start.

boot.img layout

Read layout of Android boot.img.
We now support os_version and os_patch_level.

Algorithm - binary tree traversal

1. 基本概念:

  • rooted binary tree: has 1 root, every node has {0-2} children
  • full/poper/plane binary tree: binary tree, every node has {0,2} children
  • perfect/complete binary tree: binary tree, all interior nodes has {2} children ,all leaves have same depth.

2. 递归遍历

Code speaks:

protected void traversalRecursion(Node2 root, String method) {
    if (null == root) {
        return;
    }

    switch (method) {
        case "pre-order":
            visit(root);
            traversalRecursion(root.left, method);
            traversalRecursion(root.right, method);
            break;
        case "in-order":
            traversalRecursion(root.left, method);
            visit(root);
            traversalRecursion(root.right, method);
            break;
        case "post-order":
            traversalRecursion(root.left, method);
            traversalRecursion(root.right, method);
            visit(root);
            break;
        default:
            throw new IllegalArgumentException("unknown traversalRecursion method: " + method);
    }
}

3. 迭代遍历

3.1 Pre Order Traversal: N-L-R

需要使用两个变量:current和stack

  • 先访问current的值,然后把current压栈,留着后面访问right用
  • 游标移到current.left, 继续
  • 如果current为null,但是stack不空,那么出栈,游标移到pop.right

主要思路:先一直向左冲到底,并一路压栈,见底后拿出栈顶,冲向栈顶右边,继续一直向左冲到底。
结束条件:撞见底,栈为空。
所以:需要一个冲的当前位置,需要一个栈。

protected void preOrderTraversalIteration() {
    Stack<Node2> st = new Stack<Node2>();
    Node2 current = this;
    while (true) {
        if (null != current) { //going down
            visit(current);
            st.push(current);
            current = current.left;
        } else if (!st.isEmpty()) {//visiting stack
            current = st.pop().right; //visit stack head
        } else {
            break;
        }
    }
}

3.2 In Order Traversal: L-N-R

与NLR类似

protected void inOrderTraversalIteration() {
    Stack<Node2> st = new Stack<Node2>();
    Node2 current = this;
    while (true) {
        if (null != current) { //going down
            st.push(current);
            current = current.left;
        } else if (!st.isEmpty()) {//visiting stack
            Node2 popHead = st.pop();
            visit(popHead);
            current = popHead.right;
        } else {
            break;
        }
    }
}

3.3 Post Order Traversal: L-R-N

复杂点,思路为:
一路向左冲到底,并一路压栈,
见底后冲进栈顶,如果栈顶的right是上次访问过的item,则visit栈顶然后出栈,否则冲向栈顶右边。

protected void postOrderTraversalIteration() {
    Deque<Node2> st = new LinkedList<Node2>();
    Node2 current = this;
    Node2 lastVisit = null;
    while (true) {
        if (null != current) { //going left
            st.addLast(current);
            current = current.left;
        } else if (!st.isEmpty()) {//visiting stack
            Node2 peekHead = st.getLast();
            if (lastVisit == peekHead.right) {
                visit(peekHead);
                lastVisit = st.removeLast();
            } else {
                current = peekHead.right;
                lastVisit = null; // going right
            }
        } else {
            break;
        }
    }
}

3.4 Level Order Traversal

protected void levelOrderTraversalIteration() {
    Queue<Node2> q = new LinkedList<Node2>();
    q.add(this);
    while (!q.isEmpty()) {
        Node2 current = q.remove();
        visit(current);
        if (null != current.left) {
            q.add(current.left);
        }
        if (null != current.right) {
            q.add(current.right);
        }
    }
}

Wake-On-Lan for Android

Preface

For background information about WOL, please check wikipedia:
https://en.wikipedia.org/wiki/Wake-on-LAN

The flow is: someone sends an OSI Level-2 magic packet, which contains magic header (ff ff ff ff ff ff), following by MAC addresses of target platforms.
Normally this is done by BIOS and NIC firmware, which can receive the magic packet on standby, and can power on the platform when receive the packet.
Strictly speaking, any L2 magic packet is allowed to serve as a wake up packet, but most wake up clients use UDP and discard port (port 9).

Android WakeOnLan

  1. Andorid has standard Linux suspend state,

it can enter this state by “echo mem > /sys/power/state”.
In this state, system follows standard WOL flow, NIC firmware or some low level module needs to listen to network data and wake up system.

  1. Android also has an intermediate state between awake and suspend

Android is sleeping after a call to PowerManager.goToSleep(), but sometimes application still holds wakelock, which blocking the system enters real Linux suspend state. We call it Android sleeping mode.
In this mode all userspace processes are alive, WOL features can only be implemented by user space applications.

Points need to be taken care of

  1. process needs to listen to UDP port 9 for most wake up clients. This needs the process running as root (user 0).

  2. To be more compitible, process needs to open a RAW_SOCKET to listen to all port packets, only root progresses written in C can do this. Or use JNI to call native C code.

  3. In selinux enforcing mode, additional rules are required.

  4. CTS DO NOT allow port listening.
    Android CTS case: android.security.cts.ListeningPortsTest
    We can overcome this issue by closing the listening daemon when SCREEN_ON(“android.intent.action.SCREEN_ON”), and start the listening daemon when SCREEN_OFF(“android.intent.action.SCREEN_OFF”).

Implementation

In wireshark WOL section, it recommends:

ether proto 0x0842 or udp port 9

To be more generic, we wil use:

ether proto 0x0842 or udp

TBD

java singeleton

Read this artitle first

The Most Popular solution is Bill Pugh singeleton.
Bill Pugh solution has:

  • private constructor
  • private static inner Holder class ; Holder has a private static final INSTANCE
  • public static getInstance()

Example:

package org.cfig.learn;

import java.io.Serializable;

/**
 * Created by yu on 1/14/16.
 */
public class BPsingeleton implements Serializable {
    private static final long serialVersionUID = 8382046576562889523L;

    private BPsingeleton() {
    }

    public static BPsingeleton getInstance() {
        return Holder.INSTANCE;
    }

    private static class Holder { //static inner class
        private static final BPsingeleton INSTANCE = new BPsingeleton();
    }

    protected Object readResolve() {
        return getInstance();
    }
}

Java serialization

1. Preface

用途:
Java serialization objects can be used for persistence or network data exchange.

预热文档
http://stackoverflow.com/questions/5837698/converting-any-object-to-a-byte-array-in-java
http://www.journaldev.com/2452/java-serialization-example-tutorial-serializable-serialversionuid#serializable
http://tonl.iteye.com/blog/1948429
http://www.aijava.cn/13396.html
http://www.cnblogs.com/rollenholt/archive/2012/11/26/2789445.html

2. Serializable & Externalizable

java有2个类实现序列化 Serializable,Externalizable

2.1 Serializable

  • Serializable是一个marker interface
  • transient和static字段不能序列化, but, 可以重写writeObject()来覆盖默认行为
  • 序列化对象可以写到ObjectOutputStream中,oos可以从FileOutputStream/ByteArrayOutputStream创建。
  • serialVersionUID jvm通过反射来找,不同java编译器的默认serial uid实现不一样,明确指定的话避免解析失败. (jdk自带id生成器: serialver )
  • 如果singleton类实现了serializable,那么write/read就破坏了单例模式。but, 可以用readResolve()解决,在readResolve中直接返回INSTANCE

2.2 Externalizable

  • Externalizable是一个extends Serializable的接口
  • 可以更精细地控制序列化的过程,默认constructor会被调用

3. comparison with protobuf/json

read this first.

protobuf: 需要生成代码,需要proto工具生成代码,对象小,跨语言
serialization:对象大,只能用java,语言自带解析
json: 对象大,需要第三方库,跨语言

4. code speaks

CommitInfo.groovy
CommitInfo2.groovy
Serializer.groovy
CommitInfoTest.groovy

gradle - jetty and tomcat plugin

1. Jetty plugin

What is Jetty plugin:

extends the War plugin to add tasks which allow you to deploy your web application to a Jetty web container embedded in the build.

Limitation:

default jetty version is v6.x. If you need more recent version, you need “gretty”.

2. Gretty plugin

What is Gretty plugin:

Advanced gradle plugin for running web-apps on jetty and tomcat

How to use:

remove ‘jetty’ plugin first.
Then add the following into buildscript.

apply plugin: 'war'
apply from: 'https://raw.github.com/akhikhl/gretty/master/pluginScripts/gretty.plugin'

run:

gradle appRun

Now default jetty version is 9.x.
Besides, gretty also supports tomcat. Nice toy!

@WebServlet handling difference between tomcat and jetty

Section 3.8.1 of Java Servlet Specification v3.1 says:

Classes annotated with @WebServlet class MUST extend the javax.servlet.http.HttpServlet class.

So @Webservlet annotated classes which only implements javax.servlet.Servlet should not be accessible.
But:

  • Tomcat also accepts such servlet classes.
  • Jetty follows spec strictly, it will emits warnings like “xxx is not assignable from javax.servlet.http.HttpServlet” and rejects such servlets.

code_serialization

Totally 4 files

src/main/groovy/org/cfig/learn/CommitInfo.groovy
src/main/groovy/org/cfig/learn/CommitInfo2.groovy
src/main/groovy/org/cfig/learn/Serializer.groovy
src/test/groovy/org/cfig/learn/CommitInfoTest.groovy

src/main/groovy/org/cfig/learn/CommitInfo.groovy

package org.cfig.learn

import groovy.transform.ToString

/**
 * Created by yu on 1/13/16.
 */
@ToString(includeNames = true)
class CommitInfo implements Serializable {
    private static final long serialVersionUID = 42L;

    String author;
    transient String email;
    String message;
    transient String password;

    //public CommitInfo() //unreachable

    private void writeObject(java.io.ObjectOutputStream out) throws IOException {
        println(this.class.name + ".writeObject()");
        out.defaultWriteObject();
        out.writeUTF(email);
    }

    private void readObject(java.io.ObjectInputStream inputStream) throws IOException, ClassNotFoundException {
        println(this.class.name + ".readObject()");
        inputStream.defaultReadObject();
        email = inputStream.readUTF();
    }

}

src/main/groovy/org/cfig/learn/CommitInfo2.groovy

package org.cfig.learn

import groovy.transform.ToString

/**
 * Created by yu on 1/13/16.
 */
@ToString(includeNames = true)
class CommitInfo2 implements Externalizable {
    private static final long serialVersionUID = 5923267531712419764L;

    String author;
    transient String email;
    String message;
    transient password;

    CommitInfo2() {
        println(this.class.name + ".constructor");
    }

    @Override
    void writeExternal(ObjectOutput objectOutput) throws IOException {
        println(this.class.name + ".writeExternal()")
        objectOutput.writeObject(author)
        objectOutput.writeObject(email)
        objectOutput.writeObject(message)
        objectOutput.writeObject("***")
    }

    @Override
    void readExternal(ObjectInput objectInput)
            throws IOException, ClassNotFoundException {
        println(this.class.name + ".readExternal()")
        author = objectInput.readObject()
        email = objectInput.readObject()
        message = objectInput.readObject()
        password = "###"
    }

    //private void writeObject(java.io.ObjectOutputStream out) //unreachable
    //private void readObject(java.io.ObjectInputStream inputStream) //unreachable
}

src/main/groovy/org/cfig/learn/Serializer.groovy

package org.cfig.learn

/**
 * Created by yu on 1/13/16.
 */
class Serializer {
    public byte[] serialize(Object inObj) {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(baos);
        oos.writeObject(inObj);
        return baos.toByteArray();
    }

    public Object deserialize(byte[] inBytes) {
        ByteArrayInputStream bais = new ByteArrayInputStream(inBytes);
        ObjectInputStream ois = new ObjectInputStream(bais);
        return ois.readObject();
    }
    public void serialToFile(Object inObj, String inFile) {
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(inFile));
        oos.writeObject(inObj);
        oos.close()
    }

    public Object deserializeFromFile(String inFile) {
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(inFile));
        return ois.readObject();
    }
}

src/test/groovy/org/cfig/learn/CommitInfoTest

package org.cfig.learn

import org.junit.Before;
import org.junit.Test;

/**
 * Created by yu on 1/13/16.
 */
public class CommitInfoTest {
    CommitInfo ci = new CommitInfo();
    CommitInfo2 ci2 = new CommitInfo2();

    @Before
    public void setUp() throws Exception {
        ci.author = "good";
        ci.email = "me@good.com"
        ci.message = "some message";
        ci.password = "123456";

        ci2.author = "bad";
        ci2.email = "it@bad.com"
        ci2.message = "other message";
        ci2.password = "654321";
    }

    @Test
    public void testSerializableFile() throws Exception {
        //file
        Serializer sz = new Serializer();
        sz.serialToFile(ci, "out.file");
        def ci12 = sz.deserializeFromFile("out.file");
        println(this.class.name + ".testSerializable(), read back: " + ci12);
    }

    @Test
    public void testSerializableBytes() throws Exception {
        //bytes
        Serializer sz = new Serializer();
        byte[] aBytes = sz.serialize(ci);
        def ci13 = sz.deserialize(aBytes);
        println(this.class.name + ".testSerializable(), read back: " + ci13);
    }

    @Test
    public void testExternalizable() throws Exception {
        println("Original c2: " + ci2);
        //file
        Serializer sz = new Serializer();
        //bytes
        byte[] aBytes = sz.serialize(ci2);
        def ci21 = sz.deserialize(aBytes);
        println("Read back c2: " + ci21);
    }
}

build.gradle

apply plugin: 'groovy'

repositories {
    mavenCentral()
}

dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.4.4'
    testCompile 'junit:junit:4.12'
}