2015年2月4日 星期三

Java Thread Pool

Logdown -> http://villebez.logdown.com/posts/2014/10/16/238021
這篇主要還是延續前篇 JAVA POP3 Server 實作,改善效能問題。
其實我的問題就如同 Gossip@Openhome 的 Design Pattern: Thread Pool 模式 教學說明的第一段,如下:
「在 Thread-Per-Message 模式 中,每次請求來到,就建立一個新的執行緒,用完就不再使用,然後執行緒的建立需要系統資源,對於一個接受許多請求的情況,不斷的建立新執行緒,會導致系統 效能的降低。」
有興趣看 Thread Pool 的演進,可以看看以下這三個 Design Pattern
  1. Thread-Per-Message 模式
  2. Worker Thread 模式
  3. Thread Pool 模式
但是這裡我沒有要實作 Thread Pool 模式,我只是要了解原理跟用途就好,因為 JavaSE 5.0 以後,已經有 util 可以直接達到 Thread Pool的效果了,也就是 concurrent util
所以為了這個效能問題呢,將程式改寫,並加上 Monitor Thread 程式來監看 Thread Pool 使用情形。

不廢話,直接看程式,其他自己看 java api


...

    public void run() {
        // creating the ThreadPoolExecutor

        ThreadPoolExecutor executorPool = new ThreadPoolExecutor(50, 300, 10,
                TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(30),
                Executors.defaultThreadFactory());
        // start the monitoring thread

        MonitorThread monitor = new MonitorThread(executorPool, 5);
        Thread monitorThread = new Thread(monitor);
        monitorThread.start();

        try {
            while (keeprunning) {
                Socket clientSocket = listenSocket.accept();
                clientSocket.setSoTimeout(10000);
                clientSocket.setTcpNoDelay(true);
                executorPool.execute(new manageconnection(clientSocket));
            }
        } catch (IOException excpt) {
            log.error("Sorry ,Failed I/O:" + excpt);
        }
    }

...


MonitorThread.java


public class MonitorThread implements Runnable {
    private static final Logger log = Logger.getLogger(MonitorThread.class);
    private ThreadPoolExecutor executor;

    private int seconds;

    private boolean run = true;
    
    public void shutdown() {
        this.run = false;
    }

    public MonitorThread(ThreadPoolExecutor executor, int delay) {
        this.executor = executor;
        this.seconds = delay;
    }

    @Override
    public void run() {
        while (run) {
            log.info(String
                    .format("[monitor] [%d/%d] Active: %d, Completed: %d, Task: %d, isShutdown: %s, isTerminated: %s",
                            this.executor.getPoolSize(),
                            this.executor.getCorePoolSize(),
                            this.executor.getActiveCount(),
                            this.executor.getCompletedTaskCount(),
                            this.executor.getTaskCount(),
                            this.executor.isShutdown(),
                            this.executor.isTerminated()));
            try {
                Thread.sleep(seconds * 1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }

    }
}


output
[monitor] [50/50] Active: 1, Completed: 3, Task: 4, isShutdown: false, isTerminated: false

2014年8月22日 星期五

JAVA POP3 Server 信件內文中斷符號問題修正


接續著上一篇 JAVA POP3 Server 實作 後,
這一篇主要是修正一個很奇怪的問題。


在使用Thunderbird測試一段時間後發現一個很奇怪的問題
只要跑一段時間就會出現 "RETR 命令不成功,取回郵件時發生錯誤"

在Outlook下測試卻不會,但會有很奇怪的空白信件,
空白信件的大小甚至可以到200M,這太不尋常了

反復測試 bug track後,
發現應該是信件內有 RETR cammond 的中斷符號造成。

至於兩個 Client 對於此問題的出現不一樣的結果,
我只能說,各家對於問題處裡的方式不同。

Thunderbird遇到問題就停止收信。
Outlook則是繼續收,但有問題的信跟後續的幾封信,都會變成空白信件。

之後有試著自己處理output時遇到中斷點問題,
但後來參考了 Apache James 的 POP3Handler Source Code ,
覺得寫的比較好,就拿來用了

看程式吧~
=============================================================
為了使用 FilterOutputStream 所以要加上 BufferedOutputStream

修改前
out = new PrintWriter(mySocket.getOutputStream(), AUTOFLUSH);

修改後
outs = new BufferedOutputStream(mySocket.getOutputStream(), 1024);
out = new PrintWriter(outs, AUTOFLUSH);

=============================================================
printMail 修改,使用 Apache James 的 ExtraDotOutputStream 來處理內文斷點

修改前
private void printMail(Integer index) {
String str = "";
GridFSDBFile gfsdbf = glist.get(index - 1);
InputStream inputS = null;
BufferedReader reader = null;
try {
inputS = gfsdbf.getInputStream();
reader = new BufferedReader(new InputStreamReader(inputS));

while ((str = reader.readLine()) != null) {
out.println(str);
}

} catch (Exception e) {
log.error(ExceptionUtils.getStackTrace(e));
} finally {
try {
if (inputS != null)
inputS.close();
if (reader != null)
reader.close();
} catch (Throwable ignore) {
}
}
}

修改後
private void printMail(Integer index) {
GridFSDBFile gfsdbf = glist.get(index - 1);
InputStream inputS = null;
try {
inputS = gfsdbf.getInputStream();

ExtraDotOutputStream edouts =
                    new ExtraDotOutputStream(outs);

int numRead;
byte[] buf = new byte[8192];
while ((numRead = inputS.read(buf)) >= 0) {
edouts.write(buf, 0, numRead);
}

edouts.checkCRLFTerminator();
            edouts.flush();
           
} catch (Exception e) {
out.println("-ERR");
log.error(ExceptionUtils.getStackTrace(e));
} finally {
try {
if (inputS != null)
inputS.close();
} catch (Throwable ignore) {
}
}
}


至於 ExtraDotOutputStream 這隻程式我就不附上了,
有興趣的可以去下載 Source Code 來看

2014年8月19日 星期二

JAVA POP3 Server 實作

※2014/09/01 JAVA POP3 Server 信件內文中斷符號問題修正

這篇可能有點長,主要是利用Java ServerSocket實作POP3 Server

當然免不了要先看一下什麼是 POP3 

接著就必須要瞭解 RFC 規範
基本上要看的就是 RFC1939RFC2449 這兩份文件

RFC1939
第三點 Basic Operation (基本操作),這是必看的,裡面有講那很多規定
例如:POP3 service by listening on TCP port 110.  Commands 不分大小寫. 參數長度不能超過40字元. 換行符號必須是CRLF.....等等

POP3 session 的三個狀態在基本操作也有說明
AUTHORIZATION State
TRANSACTION State
UPDATE State

接著就是必須實作的幾個指令
USER
PASS
STAT
LIST
RETR
DELE
NOOP
RSET
QUIT

POP Client Commands 的範例表


接下來就看程式吧!!

public class POP3Server extends Thread {
private static final Logger log = Logger.getLogger(POP3Server.class);
private static final int SERVER_PORT = 110;
private static final int MAX_CLIENT = 500;
private ServerSocket listenSocket = null;

public static void main(String[] args) {
POP3Server server = new POP3Server();
server.start();
}

public POP3Server() {
try {
listenSocket = new ServerSocket(SERVER_PORT, MAX_CLIENT);
} catch (IOException excpt) {
log.error("Sorry to open port " + SERVER_PORT + ":" + excpt);
System.exit(1);
}
}

public void run() {
Socket clientSocket = null;
try {
while (true) {
clientSocket = listenSocket.accept();
int numThreads = Thread.activeCount();
log.info(numThreads);
manageconnection newhandle = new manageconnection(clientSocket);
Thread newhandlethread = new Thread(newhandle);
newhandlethread.start();
log.info("hello");

}
} catch (IOException excpt) {
log.error("Sorry ,Failed I/O:" + excpt);
}
}
}

/* follow is manage and process command class */

class manageconnection implements Runnable {
private static final Logger log = Logger.getLogger(manageconnection.class);
private static final boolean AUTOFLUSH = true;
private Socket mySocket = null;
private PrintWriter out = null;
private BufferedReader in = null;
private long totalsize = 0;
private String username = null;
private List> mList;
private Set delMark = new HashSet();

public manageconnection(Socket newSocket) {
mySocket = newSocket;
}

public void run() {
String nextline = null;
int check = 0;
String command = null;
String arg1;
try {
Thread.currentThread();
Thread.sleep(10);
} catch (Exception e) {
log.error(e);
}
try {
mySocket.setTcpNoDelay(true);
} catch (SocketException excpt) {
log.error(excpt);
}
try {
System.setProperty("line.separator", "\r\n");
out = new PrintWriter(mySocket.getOutputStream(), AUTOFLUSH);
in = new BufferedReader(new InputStreamReader(
mySocket.getInputStream())

);
out.println("+OK" + " PoWei's pop3 server");
try {
while (true) {
if (check != 3) {
try {
nextline = in.readLine();
} catch (Exception excpt) {
log.error("sorry:" + excpt);
break;
}
log.info("C: " + nextline);

if (nextline.length() < 4) {
out.println("-ERR");
} else {
command = nextline.substring(0, 4);

switch (command.toUpperCase()) {
case "CAPA":
out.println("+OK Capability list follows");
out.println("USER");
out.println("PASS");
out.println("STAT");
out.println("QUIT");
out.println("LIST");
out.println("RETR");
out.println("DELE");
out.println("UIDL");
out.println("NOOP");
// out.println("TOP");
out.println(".");
break;
case "USER":
if (check == 0) {
check = 1;
if (nextline.length() < 5) {
out.println("-ERR");
} else {
arg1 = nextline.substring(5);
username = arg1;
out.println("+OK Password required for "
+ username);
}
} else {
out.println("-ERR");
}
break;
case "PASS":
if (check == 1) {
if (nextline.length() < 5) {
out.println("-ERR");
} else {
arg1 = nextline.substring(5);
if (check(username, arg1)) {
check = 2;
mList = readmail(username);
out.println("+OK " + username
+ " has " + mList.size()
+ " messages");
} else {
out.println("-ERR sorry auth failed");
check = 0;
}
}
} else {
out.println("-ERR");
}
break;
case "STAT":
if (check == 2) {
mList = readmail(username);
out.println("+OK" + " " + mList.size()
+ " " + totalsize);
} else {
out.println("-ERR");
}
break;
case "QUIT":
out.println("+OK BYE BYE welcome to PoWei's Mail System next time");
if (check == 2) {
if (!deleteMail(delMark)) {
out.println("-ERR some deleted messages not removed");
}
}
check = 3;
break;
case "LIST":
if (check == 2) {
out.print("+OK ");
out.println(mList.size() + " messages ("
+ totalsize + " byte)");
for (int j = 1; j <= mList.size(); j++) {
out.println(j + " "
+ mList.get(j - 1).get("size"));
}
out.println(".");
} else {
out.println("-ERR");
}
break;
case "RETR":
if (check == 2) {
if (nextline.length() < 5) {
out.println("-ERR");
} else {
arg1 = nextline.substring(5);
out.println("+OK");
log.info(arg1);
printMail(Integer.valueOf(arg1));
out.println(".");
boolean st = out.checkError();
log.info(st);
}
} else {
out.println("-ERR");
}
log.info("retr finished");
break;
case "DELE":
if (check == 2) {
if (nextline.length() < 5) {
out.println("-ERR");
} else {
arg1 = nextline.substring(5);
try {
Integer index = Integer
.valueOf(arg1) - 1;
if (!delMark.contains(index)) {
delMark.add(index);
out.println("+OK message "
+ arg1 + " deleted");
} else {
out.println("+ERR message "
+ arg1
+ " already deleted");
}

} catch (Exception e) {
out.println("-ERR");
}
}
} else {
out.println("-ERR");
}
break;
case "UIDL":
if (check == 2) {
if (nextline.length() < 5) {
out.println("+OK");
for (int j = 1; j <= mList.size(); j++) {
out.println(j
+ " "
+ mList.get(j - 1).get(
"uid"));
}
out.println(".");
} else {
Integer index = null;
try {
index = Integer.valueOf(nextline
.substring(5));
out.print("+OK ");
out.println(index
+ " "
+ mList.get(index - 1).get(
"uid"));
out.println(".");
} catch (IndexOutOfBoundsException e) {
out.println("-ERR IndexOutOfBoundsException:"
+ index);
} catch (NumberFormatException e) {
out.println("-ERR " + index);
}
}
} else {
out.println("-ERR");
}
break;
case "NOOP":
if (check == 2) {
out.println("+OK");
}
break;
case "TOP":
if (check == 2) {
out.println("+OK");
} else {
out.println("-ERR");
}
break;
default:
out.println("-ERR");
break;

}
}
} else {
out.close();
in.close();
mySocket.close();
break;
}

}
} catch (NullPointerException excpt) {
log.error("sorry " + excpt);
in.close();
out.close();
mySocket.close();
}

} catch (IOException excpt) {
log.error("Failed I/O:" + excpt);
}

log.info("bye");

}

private boolean deleteMail(Set delMark) {
boolean value = true;

for (Integer index : delMark) {
try {
Map map = mList.get(index);
if (map != null) {
//TODO remove mail
}

} catch (Exception e) {
value = false;
}
}

return value;
}

private void printMail(Integer index) {
String str = "";
GridFSDBFile gfsdbf = glist.get(index - 1);
InputStream inputS = null;
BufferedReader reader = null;
try {
inputS = gfsdbf.getInputStream();
reader = new BufferedReader(new InputStreamReader(inputS));

while ((str = reader.readLine()) != null) {
out.println(str);
}

} catch (Exception e) {
log.error(ExceptionUtils.getStackTrace(e));
} finally {
try {
if (inputS != null)
inputS.close();
if (reader != null)
reader.close();
} catch (Throwable ignore) {
}
}
}

/* check your username and password */
private boolean check(String username, String password) {
//TODO auth

return false;
}

private List> readmail(String uname) {
//TODO load mail

List> ret = new ArrayList>();

return ret;
}

}

2014年7月30日 星期三

Java Regular Expression (URL To Html A Tag)

※這一篇是為了解決特定用途,非URL Regular expression的介紹

需求:將輸入字串 convert URL to Html A Tag,類似Gmail寄信,會自動將URL轉換為Hyperlink

剛開始先找到了這一篇 URL Regular expression
拿到裡面的 Pattern 之後,
我利用 JavaWorld 找到的測試程式進行測試做了些小修改

修改前 https?:\/\/[\w-\.]+(:\d+)?(\/[\w\/\.]*)?(\?\S*)?(#\S*)?
修改後 https?:\/\/[\w-\.]+(:\d+)?\/?([\w\/\.]*)?(\?[\w=&;]*)?(#\w*)?

主要是將 query string 與 hash 的部分改為不允許中文字
避免連結後接續著中文字會被截取
例如:http://aaa.bbb.cc/foo.html?q=bar豬八戒


接著,為了避免取代原本就已是A Tag Href 屬性內的URL,造成重複
例如:<a href="http://aaa.bbb.cc/">aaa</a>

所以在 Pattern 最前面加上判斷非等於(=)、雙引號(")、單引號(')字元的判斷,如下
([^(=\"')])https?:\/\/[\w-\.]+(:\d+)?\/?([\w\/\.-]*)?(\?[\w=&;\.-]*)?(#\w*)?

但是這樣會造成 Matcher group 0 在取得完整的比對結果時,最前面多出一個字元
不過目前沒有想到其他辦法可以直接排除掉,只好留待取代字串時處理
所以加上了小括號(),讓他成為 group 1


好~有了 Pattern ,接下來就直接看程式吧!!  (其實是懶得解說)

private static String replaceUrlToHtml_A_Tag(String patternStr, String inputStr) {
Pattern pattern = Pattern.compile(patternStr);
Matcher matcher = pattern.matcher(inputStr);
StringBuffer sb = new StringBuffer();
while (matcher.find()) {
String groupStr = matcher.group(0).trim();
String prefix = matcher.group(1);
if(StringUtils.isNotBlank(prefix)) groupStr = groupStr.replace(prefix, "");
String format = "%s<a href=\"%s\">%s</a>";
String replacement = String.format(format, prefix, groupStr, groupStr);
matcher.appendReplacement(sb, replacement);
}
matcher.appendTail(sb);
return sb.toString();
}



以下是我的測試文件
http://aaa.bbb.cc
https://aaa.bbb.cc
aaa http://aaa.bbb.cc
rrrhttp://aaa.bbb.cc
http://aaa.bbb.cc/ bbb
http://aaa.bbb.cc 八戒
http://regexr.com:8080/foo.html abc
http://regexr.com:8080/foo.html?q=bar&amp;abc=123 abc
http://regexr.com:8080/foo.html?q=bar&amp;abc=123#aaa abc
http://regexr.com:8080/#123 abc
www.demo.com
<a href="http://aaa.bbb.cc/">123</a>
<a href="http://aaa.bbb.cc/">aaa</a>
<a href="http://aaa.bbb.cc/">abc</a>
<span style="font-family: tahoma, arial, verdana, sans-serif;">http://regexr.com:8080/foo.html?q=bar&abc=123 abc</span>


其他參考文獻
http://www.regexr.com/ 
http://www.ewdna.com/2011/09/online-regular-expression-tester.html

2014年7月1日 星期二

模擬多執行續同時執行進行測試

/**
 * @author powei.chen
 * 模擬多執行續同時執行進行測試
 */
public class MultiThread {

long startTime;

public static void main(String[] args) {
new MultiThread();
}

public MultiThread() {
startTime = new Date().getTime() + 5000;
for (int i = 0; i < 5; i++) {
Test test = new Test(i);
test.start();
}
}

class Test extends Thread {
private int no;

public Test(int no) {
this.no = no;
try {
Thread.sleep(500);// 此處表示new是需要時間的
} catch (Exception e) {
e.printStackTrace();
}
}

public void run() {
try {
Thread.sleep(startTime - new Date().getTime());// 以此方式達到同步
//TODO 執行測試
System.out.println(no + " : " + new Date());
} catch (Exception e) {
e.printStackTrace();
}
}
}

}

2014年6月11日 星期三

Java Hibernate 關聯映射 inverse & cascade


有關於 Hibernate 比較有脈絡的介紹,可以參考這邊

用了 Hibernate 好幾年,
目前也只用到 基本配置、資料查詢 (Criteria, HQL, SQL)、基本的Object-Relational Mapping

以前很少,也很懶得去設定 RDBMS 的關聯
都只是用 Single Table 的 CRUD
對於需要關聯的 Table 就用程式做掉

所以為什麼說只用到基本的ORM
因為頂多只會配置繼承映射
也因為如此~
一直沒有去研究映射的屬性

最近剛好做到一個功能,想說試試看多對多關聯
順便也研究一下其他一對多、多對一、一對一的配置方式

基本上這些關聯的用法與使用時機,就不在這裏多做介紹
因為我認為這些可不是三言兩語可以瞭解的
有興趣的朋友可以回去翻大學時的 資料庫管理系統

=~=~=~=~=~=~=~=~=~=~=~=~=主題分隔線=~=~=~=~=~=~=~=~=~=~=~=~=

呼~終於進入主題了

先來說一下這篇主要介紹的兩個屬性 inverse & cascade
請不用浪費時間將它們貼去 google 翻譯
因為只看文字本身的意思根本很難理解他們的用途

inverse (true/false, default false),負責控制關係,默認為false,也就是關係的兩端都能控制
這個屬性是在雙向關聯時使用,如果設定為 true,則控制權為對方

例如:
A, B 物件為雙向多對多關聯,此時A的inverse設定為true,則控制權為B

cascde (default none/save-update/delete),設定關聯持久化對象的操作層級
比如我刪除一個對象,那麼跟它是多對一關係的對像也全部被刪除。


詳述請看此

兩個屬性的比較與舉例可以看此


本來想說自己舉幾個例子跟設定的
但真的很懶,有空再補上吧~


http://www.mkyong.com/hibernate/hibernate-cascade-example-save-update-delete-and-delete-orphan/
http://docs.jboss.org/hibernate/orm/4.3/manual/en-US/html/

2014年5月16日 星期五

Java 泛型 interface

這篇要介紹的是 interface 使用泛型

至於泛型的定義,這裡有詳細的介紹
http://openhome.cc/Gossip/Java/
http://my.oschina.net/roockee/blog/165378

好~進入主題
前陣子工作上,在寫郵件代收程式
遇上了 POP3, IMAP 的 UID (String, long) 型別不同問題
(有興趣的人可以自己去研究一下這兩個 Protocol)

又因為兩個收信方式的處理有些微的不同,所以使用 Interface 的方式處理


首先,我定義一個 MailReceiver 的 Interface

public interface MailReceiver<T>{

public void init(Folder inbox);

public void doSomething(T uid);

public T getUID(Message msg) throws MessagingException;
}


接著就分別 Implement POP3, IMAP

public class POP3 implements MailReceiver<String> {

private POP3Folder pop3Inbox;

@Override
public String getUID(Message msg) throws MessagingException {
return pop3Inbox.getUID(msg);
}

@Override
public void init(Folder inbox) {
pop3Inbox = (POP3Folder) inbox;
}

@Override
public void doSomething(String uid) {
// TODO Auto-generated method stub

}

}


public class IMAP implements MailReceiver<Long> {

private IMAPFolder imapInbox;

@Override
public Long getUID(Message msg) throws MessagingException {
return imapInbox.getUID(msg);
}

@Override
public void init(Folder inbox) {
imapInbox = (IMAPFolder) inbox;
}

@Override
public void doSomething(Long uid) {
// TODO Auto-generated method stub

}

}


OK, 依照上面的範例
可以看到我不用在像 inbox 變數一樣,要做型別 cast
也避免了會有 ClassCastException 的問題

所以,同樣寫法,我也可以將 inbox 以泛型的寫法改寫,如下

public interface MailReceiver<F,T> {

public void init(F inbox);

public void doSomething(T uid);

public T getUID(Message msg) throws MessagingException;
}


public class IMAP implements MailReceiver<IMAPFolder, Long> {

private IMAPFolder imapInbox;

@Override
public Long getUID(Message msg) throws MessagingException {
return imapInbox.getUID(msg);
}

@Override
public void init(Folder inbox) {
imapInbox =  inbox;
}

@Override
public void doSomething(Long uid) {
// TODO Auto-generated method stub

}

}