發表文章

目前顯示的是 十月, 2012的文章

Hadoop single node setup on Ubuntu

I tried to figure out how to use Hadoop and HDFS for a while. But the information on their official site is very scattered and out-of-updated. Here are some notes I did.
A clean ubuntu 10.04 LTS build.Download hadoop package from here.The Hadoop's versioning rule is very confusing.1.0.x is stable version1.1.x is beta version2.x.x is alpha version0.23.x is similar to 2.x.x but missing Name Node HAI tried to ignore all other version started with 0.2x. I just use 1.0.4 directly.Download the KEYS in the root directoryDownload the from hadoop_1.0.4-1_i386.deb(or it's x64 version) and it's asc file from hadoop-1.0.4 folder.Check the integrityrun `gpg --import KEYS`run `gpg --verify hadoop_1.0.4-1_i386.deb.asc`You should see
mac@mac-ubuntu:~/projects/hadoop$ gpg --verify hadoop_1.0.4-1_i386.deb.asc
gpg: Signature made Thu 04 Oct 2012 01:04:55 PM PDT using RSA key ID ECB31663
gpg: Good signature from "Matthew Foley (CODE SIGNING KEY) <mattf@apache.org>"
gpg: WARNING: Th…

TrueCrypt

TrueCrypt is a good stuff that you can encrypt your data to a virtual disk. Which is actually a file resides in your regular file system. And that file can be put in your Dropbox folder, so your data can be stored in "cloud" securely.

AES encryption/decryption

Encryptionopenssl enc -e -in original_file -out original_file.aes -aes256 -k passwordDecryptionopenssl enc -d -in original_file.aes -out original_file.out -aes256 -k password
AES size: original file size + 1, then padding to 16bytes, then add 16 e.g. 1 117 bytes 117 + 1 padding to 16 bytes => 128 bytes 128 bytes + 16 = 144 bytes
e.g. 2 127 bytes 127 + 1 padding to 16 bytes => 128 bytes 128 bytes + 16 = 144 bytes
e.g. 3 128 bytes 128 + 1 padding to 16 bytes => 144 bytes 144 bytes + 16 = 160 bytes
It's irrelevant to the length of password.

Lucene

http://lucene.apache.org/core/3_6_1/demo.html

CLASSPATH

OKexport CLASSPATH=/home/mac/xxxx/xxx/xxx.jar:/home/mac/yyyy/yyy/yyy.jarexport CLASSPATH=/home/mac/xxxx/xxx/*:/home/mac/yyyy/yyy/*Not OKexport CLASSPATH=/home/mac/xxxx/xxx/*.jar:/home/mac/yyyy/yyy/*.jarexport CLASSPATH=/home/mac/xxxx/xxx/:/home/mac/yyyy/yyy/
http://lucene.apache.org/core/3_6_1/demo2.html

Need to detect doc language and change to use correct analyzer. Create Index open an directory to put index files (dir)new an Analyzer (analyzer)new an IndexWriterConfig (iwc)do some settings on IndexWriterConfiguse dir and iwc to new a IndexWriter (writer)add documentsnew a Document (doc)add several fieldsnew a Field (pathField)Field pathField = new Field("path", file.getPath(), Field.Store.YES, Field.Index.NOT_ANALYZED_NO_NORMS);pathField.setIndexOptions(IndexOptions.DOCS_ONLY);doc.add(pathField);new a NumericField (modifiedField)NumericField modifiedField = new NumericField("modified");modifiedField.setLongVal…