Cloud Computing

1.MapReduce
Map -> 將大量的資料先分割多份資料(dispatch),這些資料再送出去處理(map function)
Map可以大量被parallelize
Reduce -> 將map function所處理的結果,依據相同的key排序後,再重新整合起來
Reduce不容易被parallelize,但由於有多個不同的整體task(先Map再Reduce),所以不同的Reduce也是可以放在不同的server上執行,並不會犧牲太多的parallelism

JobTracker:用來處理由用戶丟進來的所有Job(Job會被分割成Map與Reduce),會monitor task與TaskTracker的狀況
TaskTracker:執行Mapper與Reducer

2.DFS:包含GFS, Hadoop, CloudStore
將檔案切割成固定大小的chunk(或block),檔案就是會切割成各個block存放在不同的datanode server中
使用block而非用檔案本身,主要是同一個檔放在不同的datanode可以增加IO速度
同一block可以會被replicate到不同datanode中,增加可用性
分namenode與datanode
namenode儲存所有檔名與各個block的對應,可能會有Single Point Failure
當block被讀出時,會被check block是否正確(使用CRC32),若有問題則會從另一個datanode中讀出(此時該block應該會被重新replicate一份吧?)
3.Table: BigTable, HBase, HyperTable
主要是用在儲存structure data
使用row-key, coulmn-family:qualfier, column-family:qualfier, timestamp為結構
row key即是row的值,代表一個資料
coulmn-family:qualfier的qualfier也是該column的值
coulmn-family需要先定義,表示有哪一些的column
row, column-family, timestamp都會被indexing
資料也會有version(同一個row可以有許多的version)
儲存的方式也是有一個master server與Tablet server
Master server儲存row range放在哪個tablet server中
當增加row時,會由master server決定要放在哪個tablet server中,但若是超過row數目,row range就會被split成兩個,一個留在原來的tablet server,另一個就會被移到另外的tablet server中
4.Coordination: Google Chubby, Hadoop ZooKeeper
多台機器form成一個coordination system
多台機器會挑選出一個出一個leader
每台機器的資料都是一致的,且存放在memory
read access會由當地機器處理
write access會由leader處理,並確保順序正確

5.Jute(bridge legacy data into map-reducible data)
6.HOD(Hadoop On Demand, For effective cluster sharing)
Torque
(Torque– Use the Torque node health mechanism to disable/fix ‘sick’ nodes• Great reduction in amount of support issues• Address problems in bulk)

Maui

Nagios– Usual stuff– Custom hooks into Torque

Simon– Yahoo!’s distributed cluster and application monitoring tools– Similar to Ganglia– On the roadmap to be open sourced

Hadoop相關

Hadoop相關

1.ZooKeeper (Yahoo, Apache)

2.CloudStore

3.Pig (Yahoo)

4.HBase (Powerset.com)

5.JACL (from IBM)

6.Mahout

7.Tashi

8.Hive

動態修正網頁

greasyspoon
http://greasyspoon.sourceforge.net/

隱私匿名

是否有機會
Browser->Proxy->Tor->自動選擇的open proxy
Tor可否用來做當成Wifi的抵禦工具
===========================
Tor
http://tor.eff.org
JAP
http://jackbin.blogspot.com/2008/03/jap.html
可使用JonDoFox(Firefox的JAP版)
Hotspot Shield
http://anchorfree.com
SmartHide
http://www.smarthide.com
無界瀏覽
http://www.wujie.net

======================
介紹網站
http://jackbin.blogspot.com/2007/09/tor.html
http://jackbin.blogspot.com/2008/03/jap.html

Proxy Server

Polipo
http://www.pps.jussieu.fr/~jch/software/polipo/
Privoxy
http://www.privoxy.org/user-manual/seealso.html
Squid
http://www.squid-cache.org/

Oops
http://zipper.paco.net/~igor/oops.eng/

proximodo
http://proximodo.sourceforge.net/

proxomitron
http://www.proxomitron.info/files/index.html

ProxyTrack
http://www.httrack.com/proxytrack/
HtTrack的sub-projects(WinHtTrack與WebHtTrack)

Cloud Computing Service Provider

1.Amazon Ec2
2.GAE
3.gogrid

支付工具

PayPal
Google Checkout
Amozon

非RDB的DB

以 BigTable 為基礎的 Apache HBase, Hypertable
Document-oriented database 類型的各式解決方案，如以 Erlang 開發的 Apache CouchDB
基於 Facebook Thrift 架構的 ThruDB與DistStore, Ruby 開發的 StrokeDB, 或 Java 為基礎的 FeatherDB
ZooKeeper
CloudStore

Amzon Web Service

Amazon Web Services
S3(Simple Storage Service),
SimpleDB,
EC2(Elastic Compute Cloud),
SQS(Simple Queue Service)
DevPay LS

Tools
1.Jets3t for (S3)
http://jets3t.s3.amazonaws.com/applications/applications.html#cockpit
2.typica
http://code.google.com/p/typica/downloads/list
3.CloudTools
http://code.google.com/p/cloudtools/

Java的多媒體衍生語言-Processing

http://www.proce55ing.net/

OpenFoundary的Java Tools

http://www.openfoundry.org/en/Search/newest-first/Page-7-30.html?searchphrase=any&searchword=java

Java字詞取出

http://rt.openfoundry.org/Foundry/Project/Download/?Queue=367

Java IDE

Cube-J https://cube-j.dev.java.net/
EasyEclipe http://www.easyeclipse.org/site/home/
DrJava http://drjava.org/
JIDE https://jide-oss.dev.java.net/

Java Functional Language

Jaskell
Scala
FuncionalJ http://functionalj.sourceforge.net/ch01.html

Java Source File Parser

QDox http://qdox.codehaus.org/

CI Tools (Continous Integration)

Hudson
CruiseControl

Gump (Apache)
Continuum (Apache)
DamageControl (not active)

Groovy AOP

Slingshot https://slingshot.dev.java.net/index.html
--> a framework to develop web application using AOP
AOP Blog http://chanwit.blogspot.com/search/label/AOP
Grail AOP

XML data framework

xaware (www.xware.org)
talend (http://www.talend.com/)
pentaho (http://www.pentaho.com/)

SmartClient
wicket

Functional Language

Scala

Traits
Featherweight Java http://afp.sourceforge.net/entries/FeatherweightJava.shtml
Self http://research.sun.com/self/
Cecil

Aspect相關

AspectJ
AspectWerkz
CaesarJ
HyperJ

Java Free Database

hsqldb --> good performance
db4o
mySQL
Mckoi
JavaDB(Derby)
SQLite
One$DB http://db.daffodilsw.com/Daffodil-DB-Open-Source/one$db-faqs.html

performance testing result --> http://www.polepos.org/

Naked Object Frameworks

1.Naked Objects http://www.nakedobjects.org/home/index.shtml --> Good?
Swing App. & Web
Apache License 2.0
2.Roma Framework http://www.romaframework.org/ --> Good?
Web App.
Apache License
3.OpenXava

Domain Object Explorer https://doe.dev.java.net/
JMatter(use Naked Objects) http://www.jmatter.org/
Swing App.
GPL or commercial
Trails http://www.trailsframework.org/
Web App.
GenericRCP https://genericrcp.dev.java.net/
Swing App.
based on SpringRCP
use RichJTable (https://richjtable.dev.java.net/)
Sanssouci http://sanssouci.sourceforge.net/
Swing App.
LGPL
Swing Bean Form Builder https://sbfb.dev.java.net/
Swing App.
Raskincenter.org ??

JavaOne2008的重點--JavaFX

JavaFX為此次JavaOne2008的主力重點。

SUN的執行副總裁Rich Green演示了JavaFX移動版本在Android手機上的應用。

JavaFX的預計發展：

• 六月- JavaFX 桌面SDK 先期版本
• 秋季- JavaFX 桌面1.0
• 2009春 - JavaFX Mobile 和JavaFX TV

預期發表的項目

JavaFX
程式管理：避免程式執行當機
Java nio：到底哪一個nio framework最好用
Multi-Thread：真討厭，卻又不得不用，如何才可以用得好？
怕object被不同的thread搞爛掉嗎？那何不讓處理同一個object都屬於一個thread呢？
Why Threading? 必定是多件事，必定會互相干擾才需要multi-threading
1.Call method, 2. Pass Object
message passing，不是data protection，可能是同一個Thread
Passing Data to other
1.Call method, 2.透過Queue(但需要可以使用File做Backup), 3.傳到另一個Process
Java Forum
JavaFlow：好用的library
Dalma, Rife
NAM(Network attached memory): http://www.terracotta.org/confluence/display/explore/Demo
OpenSpace, Hadoop, GigaSpace, JavaSpace, GridGain, MogileFS(from memcached)

A collection of Concurrent and Highly Scalable Utilities. These are intended as direct replacements for the java.util.* or java.util.concurrent.* collections but with better performance when many CPUs are using the collection concurrently.
http://sourceforge.net/projects/high-scale-lib

Hadoop vs GFS
HBase vs Google's BigTable vs HperTable vs memcached vs C-Store vs MonetDB
1. Google GFS & BigTable
2. Hadoop & HBase/HyperTable
3. C-Store, MonetDB
4. memcached

XStream：好用的XML Serialization Library，替代jsx

memcached的衍生品
1.memcachedb
2.Togela Cache

JMF WebCam: http://ltu164.ltu.edu/itseng/version.htm

Java Template
Velocity, FreeMaker, WebMacro

為何寫此網誌？

程式設計師真可憐，每天都趕得要死。

業主更可憐！每個案子都出問題，罵不敢罵，打不敢打，只剩下拜託。

人生這麼可憐嗎？

為台灣進一份心力，讓大家找到過一點好生活，希望有能力的開發者一起共襄盛舉。