星期二, 4月 30, 2013

Google Glass review [feedly]

星期一, 4月 29, 2013

在VirtualBox clone了一份ubuntu後網路不能用!?

原因是udev太聰明了, 它在第一次開機時把原本那張網卡的MAC address記錄在/etc/udev/rules.d/70-persistent-net.rules裡, 並且命名為eth0.
用VirtualBox clone了一份並且重新產生MAC address後, udev偵測到新的網路卡, 把新的MAC address也加進這個檔, 並且命名為eth1.
但是開機時只會依照/etc/network/interfaces內的設定去enable網卡及DHCP.
內容預設是
auto eth0
iface eth0 inet dhcp

所以只會去試著開eth0, 新偵測到的eth1就不會被用到了...

解決方法有二
一, 在/etc/network/interfaces裡加入
auto eth1
iface eth1 inet dhcp

二, 把/etc/udev/rules.d/70-persistent-net.rules裡的第一個設定移掉, 把第二個後面的name改成eth0

上面兩個方法二選一, 然後重開機, 網路應該就通了... 第二個方法是比較乾淨點...

星期五, 4月 26, 2013

Cassandra, Hive, and Hadoop: How We Picked Our Analytics Stack | MarkedUp - Analytics and Insights for Windows 8

http://blog.markedup.com/2013/02/cassandra-hive-and-hadoop-how-we-picked-our-analytics-stack/


Sent from my iPhone

Cassandra, Hive, and Hadoop: How We Picked Our Analytics Stack | MarkedUp - Analytics and Insights for Windows 8

http://blog.markedup.com/2013/02/cassandra-hive-and-hadoop-how-we-picked-our-analytics-stack/


Sent from my iPhone

更多 Google Glass 硬件细节揭晓,OMAP 4430 CPU + 1GB 内存

更多 Google Glass 硬件细节揭晓,OMAP 4430 CPU + 1GB 内存
http://www.guao.hk/posts/more-google-glass-hardware-specs-revealed-omap-4430-cpu-682mb-to-1gb-of-ram.html

前几天官方已经披露了Google眼镜一些主要的参数,包括运行Android 4.0.4系统,但并未透露CPU和内存这些跟传统电脑有关的最重要的参数。开发者JayLee今天就探索到了这些细节,他说Google眼镜使用OMAP 4430 CPU,这颗CPU跟初代Kindle使用的处理器一样。Google眼镜的可用内存是682MB(通过/proc/meminfo查到可用内存位678052kb),但通过内核信息判断,估计其内置的内存容量应该是1GB,可能有一些被其它硬件设备所占用。

Via 9to5google

© musiXboy 发表于 谷奥——探寻谷歌的奥秘 ( http://www.guao.hk ), 2013. | 没有评论 | 永久链接 | 关于谷奥 | 投稿/爆料
Post tags: Google Glass, Project Glass

shared via http://feedly.com

Google 更改政策禁止 App 繞過 Google Play 自動更新

好機車,可以這樣說改就改的嗎?

Google 更改政策禁止 App 繞過 Google Play 自動更新
http://feedly.com/k/13zyGt0

Google 今天推出了 4.0.27 版本的 Google Play,乍看之下它似乎只有一些細微的改進,但若仔細觀察的話你會發現一個非常有趣的地方。Google 此番專門為那些具備 App 內自動升級功能的 App 修改了 Google Play 的政策,未來他們將禁止所有軟體繞過 Google Play 的內部系統自動更新。這一變動對大部份 App 並不會有什麼影響,但對熱門 App 之一、最近剛剛為 Android 版本添加了此項功能的 Facebook 來說,難免會有點當頭一棒的感覺。這會是個巧合嗎?答案只有 Google 自己知道啦。

經由:Engadget
引用來源:Google Play

shared via http://feedly.com

星期四, 4月 25, 2013

Google Glass 暗藏彩蛋,是開發團隊的照片

Google Glass 暗藏彩蛋,是開發團隊的照片
http://chinese.engadget.com/2013/04/26/google-glass-easter-egg/

Google 很喜歡在他們的網站或 Android 系統裡隱藏一些「彩蛋(easter egg)」,給用戶們一些小驚喜,想不到一副只有小小螢幕的 Google Glass 也有呢。一位名為 Jay Lee 的 Glass 冒險者剛剛就發現了,只要進入裝置的 Setting -> Device Info -> View Licenses 之後連按它 9 次就會彈出 Meet the team 的畫面,點按的途中你會聽到「咇咇咇」的聲音不斷增強。Meet the team 的畫面之後就會有一張拍下整個 Google Glass 開發團隊的全景照片(可能是用 Nexus 4 拍呢)。團隊的 Mike LeBeau 已確認這隻彩蛋的存在,指其目的是要向整個團隊的人員致敬。

經由:Engadget
引用來源:Jay Lee(Google+)

shared via http://feedly.com

Access MySQL db from python

It's simpler than I expect.

>>> import MySQLdb
>>> db = MySQLdb.connect("server ip", "username", "password", "db name")
>>> cursor = db.cursor()
>>> cursor.execute("sql statement")
>>> data = cursor.fetchone()

[聯合書報攤] 台大醫院拚JCI 醫護怨嘆真悲哀

■您的好友從 聯合書報攤 寄給您的文章■

UDN / 聯合書報攤 / 新新聞
[瀏覽11264-回應1-轉寄9-列印51]
台大醫院拚JCI 醫護怨嘆真悲哀
•新新聞 2013/04/24
台大醫院最近人手兩本JCI評鑑教戰秘笈,內容涵括題庫答案、儀容應對乃至「準備茶水及甜點」招待等等,無微不至。醫護人員則痛罵,評鑑增加負擔、壓縮其他作業時間,光有文書表單,「反而掩蓋問題!」
【文╱張家豪】

紙上作業虛應故事,為評鑑不惜「變項造假」

讓台大醫院過去一、二個月嚴陣以待、將士用命,於上周一到六進行的JCI(Joint Commision International,國際聯合委員會)醫療評鑑終於落幕,全院不禁鬆一口氣,但期間院內抱怨從沒間斷過,「再怎麼評鑑也評不到真正的問題!」到底問題出在哪裡,全台醫院追逐的JCI評鑑,竟如此不得醫療人員認同?

兩本教戰手冊,彷彿學生備考、軍隊裝檢

四月中旬的這一周,是台大醫院第二度面對三年一次的JCI評鑑。除需備妥過去一年的資料,還需接受包括「病人照護」、「藥品管理與使用」、「品質改善與病人安全」、「治理、領導和管理」等十六個評鑑章節、三三八個標準及一二九七項衡量要素,全院上下因此操勞不已。

據不具名的台大醫師表示,院方為應付這次JCI評鑑,不僅特聘JCI顧問,也設立教育訓練課程,並在評鑑期間架設評鑑委員走到哪就報到哪的「情資中心」,隨時更新院內與委員對答狀況。

此外,台大醫院更人手兩本巴掌大小的JCI教戰秘笈,一本是黑白印刷、一百七十五頁的《JCIA手冊》;另一本則是彩色印刷、用來配合題庫,共三百五十四頁的《JCIA規章與流程》,如臨大敵的模樣,像極學生準備考試,其中制訂儀容打扮、如何應對等等,竟彷彿國軍部隊接受長官視導、裝備檢查。

《JCIA手冊》鉅細靡遺,不僅針對各評鑑章節,詳列二百七十五道應對題庫,也設有〈評鑑必訪查之單位、病人〉,以及指導各式表單如何填寫的〈病例記錄及審查注意事項〉等章節。而〈評鑑注意事項〉這一章中,更可看出台大醫院對JCI評鑑的高度重視及對評鑑委員的「呵護備至」。

(圖片提供╱新新聞)

表單愈多、執行愈虛,「反而掩蓋問題!」

該章第一點就指出,「記得展現5H精神:Honey(甜美的語言)、Honest(誠實的面對)、Humble(謙恭的態度)、Happy(愉悅的心情)、Honor(榮譽的信念)」;還有如「評鑑期間一律以中文(不要使用英文,除簡報╱寒暄╱讚美委員時)回答提問委員問題」、「儀表要整潔(勿蓬頭垢面),口氣要清新;醫護人員頭髮要綁起來,指甲長度需小於○•五公分」等細節。

此外,手冊也要求護理人員面對評鑑需「各備妥一、二本優良病例,當委員要求主動提供」、「避免如『我們有時會做』、『我們大部分時間都有做』等答案」、「不要在委員面前『立即改正錯誤』,否則委員會認為我們都草率做事」。更有「請體恤委員有時差,準備茶水及甜點,讓委員補充體力」等注意事項。

然而,真正令人憂心的倒還不在此,而是台大醫院為通過評鑑,竟不惜「變項造假」,刻意隱藏原來的作業方式。據台大醫師透露,各項醫院檢查及手術,會因評鑑委員即將到來而提早安排;評鑑期間則會疏散急診室病床;另外,剛從醫學院畢業,受PGY訓練(一般醫學訓練),在各科擔任住院醫師的學生,也因非正式醫師,評鑑期間要「藏起來」。

台大醫院為拚JCI評鑑,也遇到所有評鑑都會遇到的老問題──大量紙上作業。為證實達到JCI要求,醫護人員必須填勾更多的表單,診療計畫單也必須愈寫愈詳細,「病例愈來愈厚」,卻可能徒有表單但無實際執行,相當諷刺地違背《JCIA手冊》載明:「資料應真實呈現,不可捏造」。

為因應JCI評鑑的大量文書作業,及面對評鑑委員必須做的準備,不僅大大增加醫護人員的負擔,也壓縮其他醫護作業時間。虛應故事的作風,更令醫護人員痛罵:「都知道鬼在哪,卻反而掩蓋問題!」

結構性問題不先解決,評鑑勢必做假

然而,JCI評鑑並非真如洪水猛獸,台灣醫療勞動正義與病人安全促進聯盟理事、敏盛醫院品管中心主任柯紹華就認為,「JCI評鑑可以很快提升國家的醫療實力,不僅東協各國逐漸重視,未來中國若認真做JCI,台灣就挫著等。」

柯紹華解釋,JCI評鑑就像系統工程,是全世界都合用、認證標準化的評鑑,可確保一定的醫療品質和醫病安全。他舉例,JCI評鑑都從「流程」上做原則性規定,比如只規定醫院要確保病人運送過程的安全,院方則需從條文展開流程規劃,如擬定病人運送辦法及填寫病人狀況表單等。

但若發生異常狀況,柯紹華舉例,如病人的氧氣筒在運送過程中氧氣不足或耗盡,這就考驗醫院領導者,是否願意弄清原因,同樣從流程根本改善問題,比如一舉改變醫院為省成本、氧氣筒用完才可填裝的規定,而不只是在清單上新增一個選項。「JCI核心精神就是教你從內部自我檢討的正確方法。敏盛醫院人力相對於大醫院來得短缺,更需要流程設計跟科學化的系統工具,來避免環節出錯。」

然而,敏盛雖是台灣第一個申請JCI評鑑的醫院,卻在去年底決定暫緩第三次評鑑。柯紹華指出,最重要的原因之一,就是不敵台灣醫療大環境造成的結構性問題。在醫護人員過勞、出走這些問題解決前,JCI評鑑勢必做假。

(圖片提供╱新新聞)

錯誤思維演爛戲,「在台灣做下去很難!」

柯紹華強調,「敏盛急診科醫師離職很多,連班表都排不出來,急診還需外科部醫師支援,你說我怎麼苛求醫療品質?我連維持日常生活都做不到的話,我再推JCI只是逼這些人走。」他因此感嘆:「其實我覺得JCI要在台灣做下去很難!」

此外,柯紹華也指出,「台灣醫院總認為JCI是獎項,就像餐廳非拿到米其林三星不可。」而台灣醫策會評鑑則規定必須通過才能領健保給付,「就像東方思維,不考試就不念書,但考試就會有比較,JCI的邏輯則建立於西方教育,沒有一定的標準,而是必須誠實面對自己,真的比昨天好就算進步,所以你說什麼他都願意相信,這是東方遇到西方的文化衝突。」

從而,JCI與醫策會評鑑建立在不同的價值觀上,台灣醫院若未分清楚這點,很可能會拿對醫策會評鑑所做的準備及思維用在JCI,「這就是為什麼大家認為JCI都在做假,如果他今天只有考八十分的程度,你卻要求一百分,他當然要做假,但事實上JCI從沒要求一百分。」柯紹華說。

據柯紹華轉述,JCI總部人員對前衛生署長楊志良指出台灣JCI評鑑一半以上做假僅表示,「謝謝他點出問題,台灣放棄了真正改善流程,實實在在自我成長的機會。」然而,台大醫院剛落幕的JCI評鑑爛戲,卻不會是台灣最後一齣。

看更多精彩文章

星聞賽事挖挖挖∼分享娛樂好文,就抽300元7-11禮物卡!

【完整內容請見《新新聞》1364期;訂閱新新聞電子版

什麼叫「進步的社會」?

什麼叫「進步的社會」?
http://mhperng.blogspot.com/2013/04/blog-post_24.html

       左邊這一張照片出自聯合報的一則新聞「40企業老闆  響應刷馬桶」,照片裡的人物都是大老闆,從右起依序是阿瘦皮鞋董事長羅榮岳、全聯實業董事長林敏雄與台灣美化協會理事長徐重仁。他們在 4/21 這個禮拜天特地到陽明山國小帶頭清掃廁所,一方面是希望台灣的年輕人懂得「成功從彎腰做起」,不要好高鶩遠,不肯吃苦;一方面是希望通過公益活動傳遞美好的感覺。據說,當天很多大官和大老闆都「穿著短袖短褲,赤腳徒手力行清掃任務。」據說,徐重仁還表示「要找對方法做事」。

        其實這是一件好事,應該對所有參與者先鼓掌肯定。但是,他們有沒有「找對方法做事」?

        仔細看看左邊這一張照片,你有沒有發現怪異、不合理的地方?看到沒?左邊的照片裡真的大家都穿短褲和拖鞋耶!

       穿拖鞋掃廁所有什麼問題?假如你不認為這樣子有問題,看看下面這兩張照片,再跟上面的照片兩相比較,你有沒有看出哪�有問題?


        上面這兩張照片取自英國一家專業清掃廁所的公司的網站,你可以看到人家有全套制服、手套、方便的工具和口罩、眼罩、帽子。如果你去這一家公司應徵清潔員的工作,你上下班可以穿自己最喜歡的服裝,到了公司換上制服,清掃廁所省力而有效,也不用怕弄髒衣服,不會因為臭味而痛苦;做完工作脫下制服,換上最時髦的打扮回家。喔!剛剛穿的那一套臭得要死的制服呢?公司會統一拿去洗,不需勞你費心。

        這叫「有尊嚴的工作」。

        在機器人會打掃廁所之前,每一個社會都需要有人掃廁所。進步的社會裡老闆會給員工合適的穿著、配件和工具,減輕他們工作場所的不快,讓員工覺得老闆在乎他的感受,讓員工覺得自己在老闆和客戶心目中是一個有尊嚴的人,而不是競爭不過別人只好低聲下氣地做低賤的工作──就像圖片中英國掃廁所的工人。不進步的社會裡,老闆會叫員工用刻苦的態度忍受工作上的各種痛苦,讓他們跟祖父輩一樣穿著短褲和拖鞋去清掃廁所,而且希望員工以此自豪,覺得很光榮。

        當台灣的老闆和媒體嘴罵年輕人不肯吃苦時,我總是傻眼!假如要我孫子跟我祖父一樣地吃苦,那人類社會的「進步」到底是什麼意思?

        我祖父窮,死的時候沒留下任何東西,甚至還沒過世就因病無法工作,由我爸在十五歲那年扛起全家的生計。我爸一輩子省喫儉用,賺的錢幾乎都留給我們兄弟。我也省喫儉用,準備把爸爸留給我的都留給子女,再把我勤儉省下來的一起留給他們。我們兩代這麼辛苦,為的是什麼?為了讓孩子免除沒意義的痛苦和勞務,把心力和時間盡量用來做有意義的事。

        我的專業科目叫自動控制,我們開的科目有一門叫「servo mechanism」,意思就是「機器僕人」(machine servant),它的目的是讓人免除沒必要的勞務。 

       我們每一代努力的目標之一就是讓下一代可以盡量免除沒必要的痛苦和勞務,把時間與精力用來做有意義的事,而媒體和大老闆們卻罵年輕人不肯吃苦。這些人到底以為「社會進步」的目的是什麼?

        機器還沒辦法完全取代人類,為了節能不該過分追求自動化,而且也不該輕賤勞動力的意義和價值。所以,每一個社會都還是有工匠,也不該輕視木匠。

        但是,既然你看得起他,就不要折磨他。在歐洲,工匠帶口罩來避免吸入木屑、粉塵,戴耳罩減少噪音,帶眼罩避免木屑、粉塵跑進眼睛或傷害眼睛。給他合適的配備,讓他用更愉快,更少痛苦的方式做你祖父做過的事,這叫做「進步的社會」。

        不只這樣,政府還立法,要求老闆一律要照規定善待勞工,給予合適的配備和尊嚴。







        在台灣呢?三十年前是左邊照片裡那樣備嘗艱苦的工作環境,現在呢?還是差不了多少,就像右邊那張照片!不只這樣,政府還反對修訂法令來保障勞工的勞動條件,整天叫御用的經濟學者恐嚇我們:你們不用核電就會變窮,不忍受剝削與苛刻的勞動條件就要失業!

        面對這群無能的官員和學者,我真的忍不住要罵「XXX」!

        什麼叫「進步的社會」?就是讓每一代的人在做同一種工作的時候愈來愈少痛苦和不快!什麼叫落伍的社會?就是讓孫子忍受跟祖父那一代一樣的痛苦,卻還要他們自以為豪!

shared via http://feedly.com

Hadoop之父Doug Cutting

Hadoop之父Doug Cutting
http://www.programmer.com.cn/15929/

文/徐威龙

生活中,可能所有人都间接用过他的作品,他是Lucene、Nutch 、Hadoop等项目的发起人。是他,把高深莫测的搜索技术形成产品,贡献给普罗大众;还是他,打造了目前在云计算和大数据领域里如日中天的Hadoop。他是某种意义上的盗火者,他就是Doug Cutting。

从实习生做起

1985年,Cutting毕业于美国斯坦福大学。他并不是一开始就决心投身IT行业的,在大学时代的头两年,Cutting学习了诸如物理、地理等常规课程。因为学费的压力,Cutting开始意识到,自己必须学习一些更加实用、有趣的技能。这样,一方面可以帮助自己还清贷款,另一方面,也是为自己未来的生活做打算。因为斯坦福大学座落在IT行业的“圣地”硅谷,所以学习软件对年轻人来说是再自然不过的事情了。

Cutting的第一份工作是在Xerox做实习生,Xerox当时的激光扫描仪上运行着三个不同的操作系统,其中的一个操作系统还没有屏幕保护程序。因此,Cutting就开始为这套系统开发屏幕保护程序。由于这套程序是基于系统底层开发的,所以 其他同事可以给这个程序添加不同的主题。这份工作给了Cutting一定的满足感,也是他最早的“平台”级的作品。

可以说,Xerox对 Cutting后来研究搜索技术起到了决定性的影响,除了短暂的在苏格兰工作的经历外,Cutting事业的起步阶段大部分都是在Xerox度过的,这段 时间让他在搜索技术的知识上有了很大提高。他花了四年的时间搞研发,这四年中,他阅读了大量的论文,同时,自己也发表了很多论文,用Cutting自己的 话说——“我的研究生是在Xerox读的。”

尽管Xerox让Cutting积累了不少技术知识,但他却认为,自己当时搞的这些研究只是纸 上谈兵,没有人试验过这些理论的可实践性。于是,他决定勇敢地迈出这一步,让搜索技术可以为更多人所用。1997年底,Cutting开始以每周两天的时间投入,在家里试着用Java把这个想法变成现实,不久之后,Lucene诞生了。作为第一个提供全文文本搜索的开源函数库,Lucene的伟大自不必多言。

Hadoop的诞生

之后,Cutting再接再厉,在 Lucene的基础上将开源的思想继续深化。2004年,Cutting和同为程序员出身的Mike Cafarella决定开发一款可以代替当时的主流搜索产品的开源搜索引擎,这个项目被命名为Nutch。在此之前,Cutting所在的公司 Architext(其主要产品为Excite搜索引擎)因没有顶住互联网经济泡沫的冲击而破产,那时的Cutting正处在Freelancer的生涯 中,所以他希望自己的项目能通过一种低开销的方式来构建网页中的大量算法。幸运的是,Google这时正好发布了一项研究报告,报告中介绍了两款 Google为支持自家的搜索引擎而开发的软件平台。这两个平台一个是GFS(Google File System),用于存储不同设备所产生的海量数据;另一个是MapReduce,它运行在GFS之上,负责分布式大规模数据。基于这两个平台,Cutting最引人瞩目的作品——Hadoop诞生了。谈到Google对他们的“帮助”,Cutting说:“我们开始设想用4~5台电脑来实现 这个项目,但在实际运行中牵涉了大量繁琐的步骤需要靠人工来完成。Google的平台让这些步骤得以自动化,为我们实现整体框架打下了良好的基础。”

说起Google,Cutting也是它成长的见证人之一,这里有一段鲜为人知的故事。早在Cutting供职于Architext期间,有两个年轻人曾去拜访这家公司,并向他们兜售自己的搜索技术,但当时他们的Demo只检索出几百万条网页,Excite的工程师们觉得他们的技术太小儿科,于是就在心里鄙 视一番,把他们给送走了。但故事并未到此结束,这两个年轻人回去之后痛定思痛,决定自己创业。于是,他们开了一家自己的搜索公司,取名为Google。这两个年轻人就是Larry Page和Sergey Brin。在Cutting看来,Google的成功主要取决于,反向排序之后再存储的设计和对自己技术的自信。

让“开源”影响世界

出于对时间成本的考虑,在从Architext离职四年后,Cutting决定结束这段Freelancer的生涯,找一家靠谱的公司,进一步完善 Hadoop的性能。他先后面试了几家公司,其中也包括IBM,但IBM似乎对他的早期项目Lucene更感兴趣,至于Hadoop则不置可否。就在此 时,Cutting接受了当时Yahoo!搜索项目负责人Raymie Stata的邀请,于2006年正式加入Yahoo!。在Yahoo!,有一支一百人的团队帮助他完善Hadoop项目,这期间开发工作进行得卓有成效。 不久之后,Yahoo!就宣布,将其旗下的搜索业务的架构迁移到Hadoop上来。两年后,Yahoo!便基于Hadoop启动了第一个应用项目 “webmap”——一个用来计算网页间链接关系的算法。Cutting的时任上司(后为Hortonworks CEO)Eric Baldeschwieler曾说:“在相同的硬件环境下,基于Hadoop的webmap的反应速度是之前系统的33倍。”

虽然 Hadoop的表现惊艳,但在当时并非所有公司都有条件使用,与此同时,用户需求却在日益增加。有些大公司(如银行、电信公司、大型零售商等)只关注于产品,却不想在技术工程和咨询服务上过多投入,它们需要一个可以帮助其解决问题的平台,这就是Cutting后来跳槽到Cloudera的初衷。从某种程度上说,Cloudera就是这么一个为那些在咨询和技术上有需求的公司提供服务的平台。它的客户大多来自于传统行业,希望通过Hadoop来处理之前只能 被直接抛弃的大规模数据。现在,除了这些传统行业之外,Yahoo!、Facebook、eBay、LinkedIn等公司都在使用Hadoop,用 Cutting的话说,他们的团队被“无形之中扩大了”。

目前,Cutting的目标是把Hadoop发展成云计算领域的RedHat。 “我从来没有想过,除了搜索引擎,Hadoop的作用还能在其他方面有所发挥,它今天所受到的关注程度,已超过了我之前的所有想象”。谈到成功,Cutting认为他的成功主要归功于两点,一是对自己工作的热情(Cutting在大学时就开始做Infrastracture类的程序,还用 Lisp为Emacs贡献过代码,他非常喜欢自己的程序被千万人使用的感觉);二是目标不要定得过大,要踏踏实实,一步一个脚印。

 

本文选自《程序员》杂志2013年4期,未经允许不得转载。如需转载请联系 market@csdn.net

《程序员》2013年杂志订阅送好礼活动火热进行中

shared via http://feedly.com

memkeys:用 C++ 寫的 mctop (memcache top)

memkeys:用 C++ 寫的 mctop (memcache top)
http://blog.gslin.org/archives/2013/04/25/3308/memkeys%ef%bc%9a%e7%94%a8-c-%e5%af%ab%e7%9a%84-mctop-memcache-top/

在「mctop:memcache top」介紹過由 Etsy 所開發的 memcache top 工具 mctop

這套軟體用 Ruby 寫,其實就是個 sniffer + packet analyzer,但這套軟體有效能問題。在流量很高的時候無法處理所有封包,而變成 sampling 類型的監控。

Tumblr 用 C++ 新寫了一個版本,叫做 memkeys。依照軟體的說明,在 1Gbps 滿載時 mctop 約 50% 到 75% 的 packet drop (sampling rate 約 25% 到 50%),而 memkeys 只有 3% packet drop (sampling rate 約 97%):「Open Source – Memcache Top」。

This was originally inspired by mctop from etsy. I found that under load mctop would drop between 50 and 75 percent of packets. Under the same load memkeys will typically drop less than 3 percent of packets. This is on a machine saturating a 1Gb network link.


效能好不少 :p

Related Posts:

mctop:memcache top Facebook 的 Memcache 架構… 網路陸陸續續恢復了… HiNet 到 Google 改走國際線路,packet loss rate 反而降下來… 送出 ooxx HTTP Header 提升安全性…

shared via http://feedly.com

星期二, 4月 23, 2013

进程与线程的一个简单解释

进程与线程的一个简单解释
http://feedly.com/k/17Qml4a

进程(process)和线程(thread)是操作系统的基本概念,但是它们比较抽象,不容易掌握。

最近,我读到一篇材料,发现有一个很好的类比,可以把它们解释地清晰易懂。

1.

计算机的核心是CPU,它承担了所有的计算任务。它就像一座工厂,时刻在运行。

2.

假定工厂的电力有限,一次只能供给一个车间使用。也就是说,一个车间开工的时候,其他车间都必须停工。背后的含义就是,单个CPU一次只能运行一个任务。

3.

进程就好比工厂的车间,它代表CPU所能处理的单个任务。任一时刻,CPU总是运行一个进程,其他进程处于非运行状态。

4.

一个车间里,可以有很多工人。他们协同完成一个任务。

5.

线程就好比车间里的工人。一个进程可以包括多个线程。

6.

车间的空间是工人们共享的,比如许多房间是每个工人都可以进出的。这象征一个进程的内存空间是共享的,每个线程都可以使用这些共享内存。

7.

可是,每间房间的大小不同,有些房间最多只能容纳一个人,比如厕所。里面有人的时候,其他人就不能进去了。这代表一个线程使用某些共享内存时,其他线程必须等它结束,才能使用这一块内存。

8.

一个防止他人进入的简单方法,就是门口加一把锁。先到的人锁上门,后到的人看到上锁,就在门口排队,等锁打开再进去。这就叫"互斥锁"(Mutual exclusion,缩写 Mutex),防止多个线程同时读写某一块内存区域。

9.

还有些房间,可以同时容纳n个人,比如厨房。也就是说,如果人数大于n,多出来的人只能在外面等着。这好比某些内存区域,只能供给固定数目的线程使用。

10.

这时的解决方法,就是在门口挂n把钥匙。进去的人就取一把钥匙,出来时再把钥匙挂回原处。后到的人发现钥匙架空了,就知道必须在门口排队等着了。这种做法叫做"信号量"(Semaphore),用来保证多个线程不会互相冲突。

不难看出,mutex是semaphore的一种特殊情况(n=1时)。也就是说,完全可以用后者替代前者。但是,因为mutex较为简单,且效率高,所以在必须保证资源独占的情况下,还是采用这种设计。

11.

操作系统的设计,因此可以归结为三点:

(1)以多进程形式,允许多个任务同时运行;

(2)以多线程形式,允许单个任务分成不同的部分运行;

(3)提供协调机制,一方面防止进程之间和线程之间产生冲突,另一方面允许进程之间和线程之间共享资源。

(完)

文档信息

版权声明:自由转载-非商用-非衍生-保持署名 | Creative Commons BY-NC-ND 3.0 原文网址:http://www.ruanyifeng.com/blog/2013/04/processes_and_threads.html 最后修改时间:2013年4月24日 11:50 付费支持: |

[广告] GreenVPN是一款运营多年的绿色、稳定、安全的VPN产品,帮助您轻松浏览Facebook、Youtube、Twitter,使用非常简单,并且提供免费试用,值得推荐。

shared via http://feedly.com

Xperia Z Joins Its Sibling In the AOSP Mix

Xperia Z Joins Its Sibling In the AOSP Mix
http://feedly.com/k/12Gphkr

The Android Open Source Project (AOSP) is Google giving the entire community (manufacturers, enthusiasts, developers, etc.)  the necessary building blocks to bring what many refer to as "stock Android" (more accurately "vanilla Android") to a device. The inherent problem with this is that the manufacturers are often the roadblock to such endeavors. Too often manufacturers (like HTC, Samsung, etc.) and suppliers (like Qualcomm) all claim that they can't release certain drivers, and label them as "proprietary" so that no one can use them. Of course, since there's nothing really "new" under the sun, this just serves to hinder innovation and development. And often times manufacturers will claim it's the suppliers who are really hindering things, but who is it that chooses the suppliers? I'll let the obvious rhetorical question be obvious.

In this mix, it's refreshing to see a mainstream company attempt to shuck all of these trends and actually release the AOSP source for a device, with the Xperia S being the first non-Nexus device to be included in the AOSP device tree. This experiment ended on a positive note, with Sony moving the source for the Xperia S into their own managed GitHub repository. But Sony hasn't stopped there.

While companies like Samsung, which used to be rather developer-friendly, now moving away from being open to the community, Sony instead is welcoming them with open arms. Their latest flagship device, the Xperia Z, has joined the Xperia S with having its AOSP source files available on their GitHub. They even posted a lot of information over on their Developer World blog, listing SD Card, WiFi, Bluetooth, GPS, LED light, and sensors working (partially), and they state plans to include NFC in the future. They also have a link to the proprietary Qualcomm binaries needed in order for this to work. You can see the video below, and visit their blog post for more information.

shared via http://feedly.com

Apple Reports Q2 2013 Quarterly Results: $9.5 Billion Profit on $43.6 Billion in Revenue, Increases Dividend and Share Repurchases [Call Ended]

Apple Reports Q2 2013 Quarterly Results: $9.5 Billion Profit on $43.6 Billion in Revenue, Increases Dividend and Share Repurchases [Call Ended]
http://feedly.com/k/12cxpI4

Apple today announced financial results for the first calendar quarter of 2013 and second fiscal quarter of 2013. For the quarter, Apple posted revenue of $43.6 billion and net quarterly profit of $9.5 billion, or $10.09 per diluted share, compared to revenue of $39.2 billion and net quarterly profit of $11.6 billion, or $12.30 per diluted share in the year-ago quarter.

Gross margin for the quarter was 37.5 percent compared to 47.4 percent in the year-ago quarter, with international sales accounting for 66 percent of revenue. Apple also declared a increased dividend payment of $3.05 per share, payable on May 16 to shareholders as of the close of trading on May 13. The company currently holds $145 billion in cash and marketable securities.

In addition to the increase in the dividend payment, Apple said it will dramatically increase its share repurchase authorization to $60 billion from $10 billion. The company says it expects to spend $100 billion in cash under its capital return program by the end of 2015. Apple is now paying $11 billion per year in dividends to shareholders.

Quarterly iPhone unit sales reached 37.4 million, compared to 35.1 million in the year-ago quarter, and the company sold 19.5 million iPads, up from 11.8 million in the year-ago quarter. Apple sold just under 4 million Macs compared to 4 million in the year-ago period.

"We are pleased to report record March quarter revenue thanks to continued strong performance of iPhone and iPad," said Tim Cook, Apple's CEO. "Our teams are hard at work on some amazing new hardware, software and services, and we are very excited about the products in our pipeline."


Apple's guidance for the third quarter of fiscal 2013 includes expected revenue of $33.5-35.5 billion and gross margin between 36 and 37 percent.

Apple will provide live streaming of its Q1 2013 financial results conference call at 2:00 PM Pacific, and MacRumors will update this story with coverage of the conference call highlights.

Apple's stock price is trading up more than five percent after hours as the market reacts to the earnings news.

Conference Call Highlights and Q&A after the jump:

- "A lot of news to share with you today" regarding March quarter and capital return program.
- Halfway through fiscal 2013 year, introduced and ramped production of unprecedented number of new products.
- Revenue was more than $98 billion, net income more than $22 billion.
- 85 million iPhones, 42 million iPads. Unimaginable to us a few years ago.
- Beat our guidance but didn't meet everyone's expectations.
- Achieved incredible scale and financial success. Margins have decreased and growth has slowed.
- Revenues grew $13 billion in first half of the year. Average weekly growth slowed to 19%. Gross margins closer to levels from a few years ago.
- Year over year comparisons difficult because of incredible 2012.
- iPad Mini pushed margins down.
- Guiding to same revenue but decline in margins.
- Decline in stock price has been frustrating, but Apple remains strong and we will continue to do what we do best. Can't control exchange rates and world economies and certain cost pressures.
- Most important objective is to create innovative products.
- Continue to focus on the long term, remain very optimistic in our future.
- Participating in large and growing markets, see lots of promise in front of us.
- Great ecosystem.
- Potential of exciting new product categories.
- Smartphone market will double between 2012 and 2016, to 1.4 billion units annually. Tablets growing even faster.
- Amazing new hardware, software and services coming this fall and throughout 2014.
- Strongest ecosystem in the industry.
- Highest loyalty and customer satisfaction rates in the industry.
- Focus on the world's best products that change people's lives.
- Same company that brought the iPhone and iPad, and have a lot more surprises in the works.

- Returned $10 billion under share repurchase and dividend payments.
- Continue to generate cash in excess of our needs.
- Firmly committed to attractive returns to shareholders through business growth and return of capital.
- Return $100 billion by the end of calendar 2015.
- Concluded that investing in Apple was the best option.
- Increasing dividend by 15%, will access the debt markets.
- Appreciate input from so many shareholders. Will evaluate yearly and will continue to invest in the business.
- Disciplined but will not underinvest.

- New March quarter records for iPhone and iPad sales, new record for iTunes sales.
- Strong growth in iPhone and iPad sales.
- 37.4 million iPhones sold, up from 35.1 million y/y. 7% growth. 11.6 million in channel inventory, up 1 million sequentially. In target range of 4-6 weeks of inventory. #1 spot in U.S. smartphone market for 3-months ending in February. iPhone number 1 in Japan for all of calendar 2012 and Q4 2012. First time a non-Japanese company has achieved the number one spot for an entire year.
- IDC says iPhone is number 1 or 2 in smartphone sales in half the countries IDC tracks.
- 95% loyalty rate among iPhone owners, "substantially higher" than the competition. JD Power ranks iPhone top for 9 years in a row.
- 30,000 companies distributing iOS apps for their employees. Cisco BYOD increased iPhones connecting to its network by 50%.

- 19.5 million iPads versus 11.8 million y/y. Very strong growth in every operating segment, sales more than doubled in Greater China and Japan. iPad and iPad mini are extremely popular. Significantly more iPad minis sold this quarter. Overwhelming majority of iPad mini purchasers are first time iPad customers.
- iPad mini is now in target range of 4-6 weeks of iPad channel inventory.
- 96% of iPad customers are satisfied with their tablets.
- Delighted by the number of companies using iPad, plus adoption of iPad in emerging countries.

- Just under 4 million Macs. 2% down y/y. Global computer market contracted 14% y/y. Decline in portable sales, increase in iMac sales.
- 4-5 weeks of Mac channel inventory.

- 5.6 million iPods, iPod Touch accounts for more than half of iPods sold. iPod share of US market of MP3 players was more than 70% of market.

- Apple ecosystem continues to grow and thrive.
- Very important driver of customer satisfaction and retention.
- Sunday marks 10th anniversary of iTunes Music Store.
- Quarterly billings of $4 billion, a new record. $16 billion annual run rate. Largest digital content store in the world.
- Music stores in 119 countries, 35 million songs, movies in 109 countries, iBookstore in 155 countries. App Store in 155 countries, covering 90% of world's population. 850,000 iOS apps, 350,000 apps for iPad.
- App downloads passed 45 billion, app devs have made $9 billion total.
- App Store accounted for 74% of all app sales worldwide in March quarter.
- App Store accounted for 95% of Backflip Studio's annual $100 million in revenue.

- iOS platform offers secure environment.
- 97% of mobile malware on Android.
- iPad accounted for 82% of all North American tablet traffic in March.
- 800 apps downloaded per second in the quarter, y/y app revenue doubled.

- 300 million people using iCloud. Great new features and capabilities in the pipeline.

- $5.2 billion in retail, up 19%. Up because of iPhone and iPad sales. 402 stores, including 151 outside the US. Open 30 new stores in fiscal 2013, plus 20 store remodels. Average revenue per store $13.1 million vs $12.2 million y/y. 91 million visitors compared to 85 million y/y. 78,000 per store per week.

- $144.7 billion in cash plus short and long term marketable securities. Up $7.6 billion. $102 billion is offshore.
- $2.5 billion in dividends, concluded $1.95 billion accelerated share repurchase program. 4 million shares of Apple stock retired.
- Doubling return of capital program to $100 billion by end of 2015.
- Average of $30 billion per year.
- Dividends, share repurchases.

- Will access debt markets.
- $60 billion in share buyback, largest of any company in history. Repurchasing shares is attractive use of capital. Buy shares beginning this month. Continue to net share settle employee RSU's. $1 billion of cash annually.
- Quarterly dividend to $3.05/share from $2.65, up 15%.
- Will review annually.
- Among the largest dividend payments in the world, roughly $11 billion per year.
- Will access U.S. debt ratings over time. Generating significant cash offshore, repatriating cash would cause significant tax consequences.
- Access to attractively priced capital, efficient leverage of our balance sheet.
- Will maintain domestic liquidity.
- $30 billion would have represented all of fiscal 2012 cash flow had we repatriated our foreign earnings.
- Expect our foreign earnings and domestic borrowing to grow through 2015.
- Aim to maintain flexibility to invest while we return capital.

Q&A Session

Q: Gross margin came in at low end of guidance, can you rank for us the biggest factors that are limiting margin recovery?
A: March quarter, gross margin was 37.5% at low end of range. Had a few items that resulted in this: mix (more iPads and iPad mini into channel), changes in service policies, unfavorable adjustments. Guiding gross margin down 50 to 150 basis points sequentially. Largest reason is loss of leverage on sequentially lower revenue and a different product mix. Both of these will be partially offset by better costs in the quarter.

Q: Growth slowed in China to single digits, curious your perspective given stickiness and the fact that Apple is not in the biggest carrier there.
A: Greatest quarter for China ever, $8.8 billion including retail stores, up 11% year on year. Same as Apple's growth. Highlights included iPad growing 108% y/y, new records for sell through for iPad and iPhone. Significant y/y timing differences related to iPhone.

Last year, increased channel inventories by 2.6 million. 1.6 of that occurred in Greater China. If you look on a sell through basis, Greater China was up approximately 18%.

Going forward, we see a significant opportunity in China. It's a great market. Expect to double number of stores from 11 to 22 in less than 2 years. Added 8k iPhone point of sales to 19k. Have a plan to add more and further grow our distribution. This is too low currently.

Innovating in our online store there, adding different functionality to the store. China has an unusually high number of first time smartphone buyers. We've seen significant interest in iPhone 4 there and recently made it even more affordable to make it more attractive to those first time buyers.

Q: Competitive landscape in 2013 vs 2012? How has it surprised you?
A: Smartphone market has always been competitive. The names have changed. In the beginning, RIM was the strongest player. Today, our top competitor is Samsung married to Google. They're tough competitors but we feel that we have the best products by far. Continuing to invest in innovative products and feel really confident about our product pipeline in both hardware, software and services. We have the best ecosystem by far, plan to keep making it better. That shows up in loyalty ratings and customer satisfaction.

Q: How are you thinking about the pace of your buyback? What factors will you consider in purchase timing over next few years?
A: Begin buying shares beginning this month. Utilize remaining $58 billion by end of 2015, spanning 32 months. Accelerated share repurchase and to buy in the open market.

Q: You alluded to fall announcements. Does that mean there won't be big announcements before then or are you being more general?
A: Don't want to be more specific, but we have some really great stuff coming in the fall and across all of 2014.

Q: Talk about supply chain? Some concerns about product limitations, but getting huge relative to what you used to run. Talk about moving to new vendors?
A: We exited March quarter with no shortages while achieving inventory turns over 90, still staggering for our industry. The December was largest quarter ever in history of Apple and tech industry. Included unprecedented number of new products and ramps.

Work we do to produce truly innovative products is very hard. Challenges that we'll face in the future, but we're working very closely with our manufacturing partners and we are executing a very exciting roadmap.

Q: Revisit gross margin, comment on experiencing normal kinds of experience curves that you do in typical product ramps, can you comment on sequential decline in iPhone ASP (down $30 sequentially)? A mix of 4 and 4S relative to the 5, but can you confirm? What's the impact on margin going forward?
A: Very pleased with progress on getting cost curves down, help to offset loss of leverage and mix in next quarter, factored these cost improvements into our guidance. iPhone ASP is down sequentially because of mix. Increase in iPhone 4 mix, resulting from making iPhone 4 more affordable in many markets, also mix within iPhone 5 lineup.

Q: How do we think about margins for iPhone 4 vs 5?
A: We don't provide margins within product lines. Talked about two factors impacting gross margin on sequential basis. Leverage is largest reason for decline, but different product mix going forward on iPhone.

As you get further from a previous product launch, there is a general lower mix within a line than where it starts. You'd expect to see that on most of our products.

Q: You talked about excitement regarding large addressable markets. Talked about how strongly and confidently you feel about competitive advantage. If I step back and look at the data on the phone side, adjusted for inventory the iPhone was up low to mid single digits. Third party market data says market grew 30%. Next quarter, iPhone won't match the smartphone market growth overall. Is that a problem and is it something Apple cares about? What is the disconnect between market growth and your products?
A: Tablet numbers that we've seen from IDC, believe market declined 30% from December. We declined 15%. If that holds, we did better than the market and a nice pickup in market share. On iPad, we continue to have an ecosystem story that is unparalleled. 350,000 apps for iPad, versus a few hundred for competition. Plus overall experience and fit and finish is better than options from competition.

On phone side, the numbers you talk about, the sell-in comparisons y/y, you have to look at sell through to examine demand. We did grow channel inventory last year to catch up on iPhone 4S rollout. You have to factor that to get at sell through. If market grew by 30%, we grew less than that. The point is not lost, we do want to grow faster than the market. We don't view it as the only measure of our health. The things that are important to us in addition to market share and unit volume include customer sat (highest by far), customer loyalty and repurchase (highest by far), and ecosystem commerce which attracts developers is incredible. 3/4 dollars spent on apps are spent in our ecosystem. Usage statistics are staggering comparing iOS to other operating systems. These other things are extremely important and we're all about customer experience and enriching lives.

That said, we see an enormous number of first time smartphone buyers coming to market particularly in certain countries around the world. What we've done there is to make the iPhone 4 even more affordable and more attractive to first time buyers. We are caught up on supply/demand on the 4 and we're continuing to do that in other markets. We believe that the phone for the price point that we're offering, is an incredible value for people that gets people into our ecosystem with a phenomenal product.

Q: Comment on gross margin beyond next quarter? What does it say about long term gross margins if 36-37 is the margin when your highest margin product is at a trough for the year? I think that's what you're implying. Is there an implied margin philosophy where this is as low as you want it to get, or is this not what we should be reading?
A: Been our long term practice that we aren't providing specific guidance beyond the June quarter. Many things that can affect margins and can cause them to vary widely over time. Don't want to make a forecast, but this is how we think about gross margin.

Managing the business for long term. Willing to trade short term profits for long term potential. Launched iPod in 2001, margins were significantly below margins for Apple. Later, iPod and iTunes comprised half of Apple's revenue. Inspired us to build the iPhone. iPad mini is similar, margins significantly below the corporate average. Only make great products. Won't make cheap ones.

Many people around the world who want great products. Made great business decisions to balance unit sales and profit. Managing Apple with a long term horizon. Much revenue comes after we ship our products. Received $4 billion in revenue for services for the first time, will augment ecosystem with new services and make existing ones even better.

Q: What is your reaction to what you think of the 5-inch phone market at this time versus 3 months ago. Has anything changed in your view?
A: My view continues to be that iPhone 5 has the best display in the industry. We always strive to create the very best display. Some customers value large screen size. Others value other factors such as resolution, color quality, white balance, reflectivity, power consumption, compatibility of apps, portability. Our competitors have made some significant tradeoffs in many of these areas to ship a larger display. We would not ship a larger display iPhone while these tradeoffs exist.

Q: Did you mention new product categories?
A: One of our areas for growth are potential new categories.

Q: Would that be in the next year?
A: Didn't announce a specific time frame.

Q: Mentioned new services and improving specific services, are there any other key access that you can leverage into new services?
A: Very fortunate to have the largest and broadest ecosystem in the world. App stores in 155 countries and iTunes in more than 110 countries. Hundreds of millions of users. $16 billion run rate on iTunes Stores in billings, and $4 billion in revenue for services. Opportunity to invest in tremendously to continue to grow our businesses in those areas.

Because we are not fragmented like our competition, we can update an OS with a major release and a substantial percentage of our customers will update to our latest software. We've made that elegant and easy. Because the usage for iOS is so much higher, when we integrate things well, people use them a lot more. So, just those concepts by themselves are huge advantages from a customer experience point of view and from the metrics that you're thinking about.

Q: In terms of payments market, do you feel this is well addressed with the offerings out there?
A: I think it's in its infancy. I think it's just getting started. Just out of the starting block.

Q: What are you thinking of in the Mac market versus the PC market? Curious whether you think elasticity of demand with lower prices, what you're hearing from your customers, or how are you feeling about cannibalization potential?
A: Reason that we were down last quarter (2%), market for PCs is incredibly weak. Market is down 14% y/y, largest decline that I can remember. We sold almost 20 million iPads, and its certainly true that some of those iPads cannibalized those Macs. I don't think it was a huge number but I think it's some. The larger thing here is that on the PC side people are extending their upgrade cycles. This market isn't a dead market or a bad market by any means. Has a lot of life to it, continue to innovate in it. If anything, the huge growth in tablets may wind up benefiting the Mac. It pushes people to think about that product in a different manner. People may be more willing to buy a Mac versus a PC. Will continue to make the best personal computers. Strategy isn't changing. Great innovation like the retina display, and more great stuff planned. Area we're continuing to invest in.

Q: Competitors starting to talk about wins in the enterprise. I'm curious what you've heard from your customers, seen much displacement? What are discussions you're having with enterprise customers with regards to iPads?
A: Seem to be doing really well. iOS accounting for 77% of activations by corporate customers according to one study. Would not include BlackBerry but would include the other guys. Seem to be doing really well, I don't see recent announcements changing that. More and more people developing custom apps for their businesses on iOS to be used on iPad, we're very bullish on it.

iPad now is being used in 95% of Fortune 500, on Global 500, we're in 89%.

Q: December quarter had broad product refresh. Did you learn that you should stagger those launches over multiple quarters instead of one quarter?
A: I don't spend a lot of time looking back except to learn from it. If we could run it over, I'd announce the iMac after the turn of the year. We felt customers had to wait too long for that specific product. The iPad mini was in shortage in the December quarter, I would not have done that differently. We were able to get the mini out to millions of customers who really wanted that for the holiday. Yes, we did do a lot, our teams can do a lot. In retrospect, that one I wish we were after the turn of the year so customers wouldn't have to wait as long as they did.

Q: Are you seeing acceleration in usage rates?
A: We see incredible usage in every study we see. Vast majority coming not from Apple. Usage of iOS devices are staggering compared to competitors. A phenomenal difference. That shows the strength of the ecosystem. We are now paying our developers more than $1 billion per quarter.

Recent Mac and iOS Blog Stories
Apple Store iOS App Updated With Push Delivery and iPhone Upgrade Notifications
FocusTwist Allows iPhone Users to Capture Refocusable Lytro-style Images
Apple to Open New Retail Locations in Melbourne and Quebec on Saturday
Tweetbot for iOS Updated with Redesigned Image Viewer and New Media Timeline
Logitech Announces New FabricSkin Keyboard Folio
'FedEx Mobile' Now Lets Recipients Sign for and Manage Shipments
$15 iPhone 4 Antenna Settlement Checks Now Being Received
Apple Cuts Prices on Refurbished 2nd and 3rd Generation Cellular iPads

    

shared via http://feedly.com

星期一, 4月 22, 2013

維基百科英文版與德文版的資料庫 (條目最多的兩個語言) 從 MySQL 轉移到 MariaDB 了…

維基百科英文版與德文版的資料庫 (條目最多的兩個語言) 從 MySQL 轉移到 MariaDB 了…
http://feedly.com/k/128s6tn

維基百科官方宣佈兩個條目最多的資料庫 (英文與德文) 已經從 MySQL (有 FB patch 的版本) 轉移到 MariaDB 5.5 了:「Wikipedia Adopts MariaDB」。

維基百科的資料庫從 MySQL 4.0 升級到 MySQL 5.1 時花了不少功夫轉換 (可以想像出來,這兩個版本的差距…),然後這次再到這次的 MariaDB 5.5 就輕鬆不少。

在文章內有提到因為維基百科是 read-heavy site,大多數前端的負荷都在 squid 層擋下來,實際到後端的量則是再透過 Redismemcached 打散負荷。不過即使做了這麼多層 cache,英文版資料庫在尖峰時間還是有 50k qps 的量在跑。

尖峰時間這 50k qps 的量有 80% (也就是 40k qps) 是打散到兩台不同地點的 slave 上,平均的 query response time 是 0.2ms,95% 則是 50ms (好高),其他的 20% 是寫入 master 需求,或是因寫入 master 時需要一致性 (也就是要避免 replication lag 造成的問題)。

這次升級到 MariaDB 5.5.30 是先準備一台新機器,然後在 load balancer 上換掉其中一台 slave (先建後拆),如果 MariaDB 真的有問題也可以馬上 rollback 回來。另外用 pt-query-digest 取樣分析 query 的狀況。

這是成果:

For our most common query type, 95th percentile times over an 8-hour period dropped from 56ms to 43ms and the average from 15.4ms to 12.7ms. 50th percentile times remained a bit better with the 5.1-facebook build over the sample period, 0.185ms vs. 0.194ms. Many query types were 4-15% faster with MariaDB 5.5.30 under production load, a few were 5% slower, and nothing appeared aberrant beyond those bounds.


第一台觀察一陣子沒問題後,接下來一台一台換掉。然後發公告慶祝 :p

Related Posts:

Instagram 說明用 PostgreSQL 的五個優點… 跳過 MySQL replication 失敗的方法… 熱 MySQL 的方法… 資料庫裡的浮點數:MySQL 5.1 到 MySQL 5.5 的差異… Percona 說明關於為什麼花這麼多時間修正 MySQL 5.5.28 安全問題…

shared via http://feedly.com

大数据的存储和管理

大数据的存储和管理
http://feedly.com/k/15Cz0v8

文/陆嘉恒

任何机器都会有物理上的限制:内存容量、硬盘容量、处理器速度等,我们需要在这些硬件的限制和性能之间做出取舍,比如内存的读取速度比硬盘快得多,因此内存数据库比硬盘数据库性能好,但是内存为2GB的机器不可能将大小为100GB的数据全部放入内存中,也许内存大小为128GB的机器能够做到,但是数据增加到200GB时就无能为力了。

数据不断增长造成单机系统性能不断下降,即使不断提升硬件配置也难以跟上数据的增长速度。然而,当今主流的计算机硬件比较便宜而且可以扩展,现在购置八台8内核、128GB内存的机器比购置一台64内核、TB级别内存的服务器划算得多,而且还可以增加或减少机器来应对将来的变化。这种分布式架构策略对于海量数据来说是比较适合的,因此,许多海量数据系统选择将数据放在多个机器中,但也带来了许多单机系统不曾有的问题。

下面我们介绍大数据存储和管理发展过程中出现的四类大数据存储和管理数据库系统。

并行数据库

并行数据库是指那些在无共享的体系结构中进行数据操作的数据库系统。这些系统大部分采用了关系数据模型并且支持SQL语句查询,但为了能够并行执行SQL的查询操作,系统中采用了两个关键技术:关系表的水平划分和SQL查询的分区执行。

水平划分的主要思想就是根据某种策略将关系表中的元组分布到集群中的不同节点上,这些节点上的表结构是一样的,这样就可以对元组并行处理。现有的分区策略有哈希分区、范围分区、循环分区等。例如,哈希分区策略是将表T中的元组分布到n个节点上,可以使用统一的哈希算法对元组中的某个或某几个属性进行哈希,如hash(T.attribute1) mod n,然后根据哈希值将元组放置到不同的节点上。

在分区存储的表中处理SQL查询需要使用基于分区的执行策略,如获取表T中某一数值范围内的元组,系统首先为整个表T生成总的执行计划P,然后将P拆分成n个子计划{P1,…,Pn},子计划Pi在节点ni上独立执行,最后每个节点将生成的中间结果发送到某一选定的节点上,该节点对中间结果进行聚集产生最终的结果。

并行数据库系统的目标是高性能和高可用性,通过多个节点并行执行数据库任务,提高整个数据库系统的性能和可用性。最近一些年不断涌现一些提高系统性能的新技术,如索引、压缩、实体化视图、结果缓存、I/O共享等,这些技术都比较成熟且经得起时间的考验。与一些早期的系统如Teradata必须部署在专有硬件上不同,最近开发的系统如Aster、Vertica等可以部署在普通的商业机器上,这些数据库系统可以称得上准云系统。

并行数据库系统的主要缺点就是没有较好的弹性,而这种特性对中小型企业和初创企业是有利的。人们在对并行数据库进行设计和优化的时候认为集群中节点的数量是固定的,若需要对集群进行扩展和收缩,则必须为数据转移过程制订周全的计划。这种数据转移的代价是昂贵的,并且会导致系统在某段时间内不可访问,而这种较差的灵活性直接影响到并行数据库的弹性以及现用现付商业模式的实用性。

并行数据库的另一个问题就是系统的容错性较差,过去人们认为节点故障是个特例,并不经常出现,因此系统只提供事务级别的容错功能,如果在查询过程中节点发生故障,那么整个查询都要从头开始重新执行。这种重启任务的策略使得并行数据库难以在拥有数以千个节点的集群上处理较长的查询,因为在这类集群中节点的故障经常发生。基于这种分析,并行数据库只适合于资源需求相对固定的应用程序。不管怎样,并行数据库的许多设计原则为其他海量数据系统的设计和优化提供了比较好的借鉴。

NoSQL数据管理系统

NoSQL一词最早出现于1998年,它是Carlo Strozzi开发的一个轻量、开源、不提供SQL功能的关系型数据库(他认为,由于NoSQL悖离传统关系数据库模型,因此,它应该有一个全新的名字,比如“NoREL”或与之类似的名字)。

2009年,Last.fm的Johan Oskarsson发起了一次关于分布式开源数据库的讨论[7],来自Rackspace的Eric Evans再次提出了NoSQL的概念,这时的NoSQL主要指非关系型、分布式、不提供ACID的数据库设计模式。

2009年在亚特兰大举行的“no:sql(east)”讨论会是一个里程碑,其口号是”select fun, profit from real_world where relational=false;”。因此,对NoSQL最普遍的解释是“非关系型的”,强调键值存储和文档数据库的优点,而不是单纯地反对关系型数据库。

传统关系型数据库在处理数据密集型应用方面显得力不从心,主要表现在灵活性差、扩展性差、性能差等方面。最近出现的一些存储系统摒弃了传统关系型数据库管理系统的设计思想,转而采用不同的解决方案来满足扩展性方面的需求。这些没有固定数据模式并且可以水平扩展的系统现在统称为NoSQL(有些人认为称为NoREL更为合理),这里的NoSQL指的是“Not Only SQL”,即对关系型SQL数据系统的补充。NoSQL系统普遍采用的一些技术有:

简单数据模型。不同于分布式数据库,大多数NoSQL系统采用更加简单的数据模型,这种数据模型中,每个记录拥有唯一的键,而且系统只需支持单记录级别的原子性,不支持外键和跨记录的关系。这种一次操作获取单个记录的约束极大地增强了系统的可扩展性,而且数据操作就可以在单台机器中执行,没有分布式事务的开销。 元数据和应用数据的分离。NoSQL数据管理系统需要维护两种数据:元数据和应用数据。元数据是用于系统管理的,如数据分区到集群中节点和副本的映射数据。应用数据就是用户存储在系统中的商业数据。系统之所以将这两类数据分开是因为它们有着不同的一致性要求。若要系统正常运转,元数据必须是一致且实时的,而应用数据的一致性需求则因应用场合而异。因此,为了达到可扩展性,NoSQL系统在管理两类数据上采用不同的策略。还有一些NoSQL系统没有元数据,它们通过其他方式解决数据和节点的映射问题。 弱一致性。NoSQL系统通过复制应用数据来达到一致性。这种设计使得更新数据时副本同步的开销很大,为了减少这种同步开销,弱一致性模型如最终一致性和时间轴一致性得到广泛应用。

通过这些技术,NoSQL能够很好地应对海量数据的挑战。相对于关系型数据库,NoSQL数据存储管理系统的主要优势有:

避免不必要的复杂性。关系型数据库提供各种各样的特性和强一致性,但是许多特性只能在某些特定的应用中使用,大部分功能很少被使用。NoSQL系统则提供较少的功能来提高性能。 高吞吐量。一些NoSQL数据系统的吞吐量比传统关系数据管理系统要高很多,如Google使用MapReduce每天可处理20PB存储在Bigtable中的数据。 高水平扩展能力和低端硬件集群。NoSQL数据系统能够很好地进行水平扩展,与关系型数据库集群方法不同,这种扩展不需要很大的代价。而基于低端硬件的设计理念为采用NoSQL数据系统的用户节省了很多硬件上的开销。 避免了昂贵的对象-关系映射。许多NoSQL系统能够存储数据对象,这就避免了数据库中关系模型和程序中对象模型相互转化的代价。

NoSQL向人们提供了高效便宜的数据管理方案,许多公司不再使用Oracle甚至MySQL,他们借鉴Amzon的Dynamo和Google的Bigtable的主要思想建立自己的海量数据存储管理系统,一些系统也开始开源,如Facebook将其开发的Cassandra捐给了Apache软件基金会。

虽然NoSQL数据库提供了高扩展性和灵活性,但是它也有自己的缺点,主要有:

数据模型和查询语言没有经过数学验证。SQL这种基于关系代数和关系演算的查询结构有着坚实的数学保证,即使一个结构化的查询本身很复杂,但是它能够获取满足条件的所有数据。由于NoSQL系统都没有使用SQL,而使用的一些模型还未有完善的数学基础。这也是NoSQL系统较为混乱的主要原因之一。 不支持ACID特性。这为NoSQL带来优势的同时也是其缺点,毕竟事务在很多场合下还是需要的,ACID特性使系统在中断的情况下也能够保证在线事务能够准确执行。 功能简单。大多数NoSQL系统提供的功能都比较简单,这就增加了应用层的负担。例如如果在应用层实现ACID特性,那么编写代码的程序员一定极其痛苦。 没有统一的查询模型。NoSQL系统一般提供不同查询模型,这一定程度上增加了开发者的负担。

NewSQL数据管理系统

人们曾普遍认为传统数据库支持ACID和SQL等特性限制了数据库的扩展和处理海量数据的性能,因此尝试通过牺牲这些特性来提升对海量数据的存储管理能力,但是现在一些人则持有不同的观念,他们认为并不是ACID和支持SQL的特性,而是其他的一些机制如锁机制、日志机制、缓冲区管理等制约了系统的性能,只要优化这些技术,关系型数据库系统在处理海量数据时仍能获得很好的性能。

关系型数据库处理事务时对性能影响较大、需要优化的因素有:

通信。应用程序通过ODBC或JDBC与DBMS进行通信是OLTP事务中的主要开销。

日志。关系型数据库事务中对数据的修改需要记录到日志中,而日志则需要不断写到硬盘上来保证持久性,这种代价是昂贵的,而且降低了事务的性能。 锁。事务中修改操作需要对数据进行加锁,这就需要在锁表中进行写操作,造成了一定的开销。 闩。关系型数据库中一些数据结构,如B树、锁表、资源表等的共享影响了事务的性能。这些数据结构常常被多线程读取,所以需要短期锁即闩。 缓冲区管理。关系型数据将数据组织成固定大小的页,内存中磁盘页的缓冲管理会造成一定的开销。

为了解决上面的问题,一些新的数据库采用部分不同的设计,它取消了耗费资源的缓冲池,在内存中运行整个数据库。它还摈弃了单线程服务的锁机制,也通过使用冗余机器来实现复制和故障恢复,取代原有的昂贵的恢复操作。这种可扩展、高性能的SQL数据库被称为NewSQL,其中“New”用来表明与传统关系型数据库系统的区别,但是NewSQL也是很宽泛的概念。它首先由451集团在一份报告中提出,其主要包括两类系统:拥有关系型数据库产品和服务,并将关系模型的好处带到分布式架构上;或者提高关系数据库的性能,使之达到不用考虑水平扩展问题的程度。前一类NewSQL包括Clustrix、GenieDB、ScalArc、ScaleBase、NimbusDB,也包括带有NDB的MySQL集群、Drizzle等。后一类NewSQL包括Tokutek、JustOne DB。还有一些“NewSQL即服务”,包括Amazon的关系数据库服务、Microsoft的SQL Azure、FathomDB等。

当然,NewSQL和NoSQL也有交叉的地方,例如,RethinkDB可以看作NoSQL数据库中键/值存储的高速缓存系统,也可以当作NewSQL数据库中MySQL的存储引擎。现在许多NewSQL提供商使用自己的数据库为没有固定模式的数据提供存储服务,同时一些NoSQL数据库开始支持SQL查询和ACID事务特性。

NewSQL能够提供SQL数据库的质量保证,也能提供NoSQL数据库的可扩展性。VoltDB是NewSQL的实现之一,其开发公司的CTO宣称,它们的系统使用NewSQL的方法处理事务的速度比传统数据库系统快45倍。VoltDB可以扩展到39个机器上,在300个CPU内核中每分钟处理1600万事务,其所需的机器数比Hadoop集群要少很多。

随着NoSQL、NewSQL数据库阵营的迅速崛起,当今数据库系统“百花齐放”,现有系统达数百种之多,图1-1将广义的数据库系统进行了分类。

数据库系统的分类

图中将数据库分为关系型数据库、非关系型数据库以及数据库缓存系统。其中,非关系型数据库主要指的是NoSQL数据库,分为:键值数据库、列存数据库、图存数据库以及文档数据库四大类。关系型数据库包含了传统关系数据库系统以及NewSQL数据库。

高容量、高分布式、高复杂性应用程序的需求迫使传统数据库不断扩展自己的容量极限,这些驱动传统关系型数据库采用不同的数据管理技术的6个关键因素可以概括为“SPRAIN”,即:

可扩展性(Scalability)——硬件价格 高性能(Performance)——MySQL的性能瓶颈 弱一致性(Relaxed consistency)——CAP理论 敏捷性(Agility)——持久多样性 复杂性(Intricacy)——海量数据 必然性(Necessity)——开源

作者陆嘉恒,中国人民大学教授,博士生导师。2006年毕业于新加坡国立大学计算机科学系,获博士学位;2006-2008年在美国加利福尼亚大学尔湾分校进行博士后研究;2008年加入中国人民大学,2012年破格晋升为教授。主要研究领域包括数据库技术和云计算技术。先后在SIGMOD、VLDB、ICDE、WWW等国际重要会议和期刊上发表数据库方向的论文40多篇,主编多本云计算和大数据的教材和著作。

本文节选自《大数据挑战与NoSQL数据库技术》一书,陆嘉恒编著,由电子工业出版社出版。

shared via http://feedly.com

星期三, 4月 17, 2013

中國大陸雅虎郵箱啓動整體遷移,下一站阿里雲

想不到阿里雲搞這麼大了

中國大陸雅虎郵箱啓動整體遷移,下一站阿里雲
http://feedly.com/k/15mZocb

各位讀者如果有人用大陸雅虎的郵箱服務的話,在今天登入郵箱時有沒有收到上圖中這一份通知呢?實話說一切來得真是有點突然,可能誰也不曾想到這一老牌電郵服務會以如此的方式離開中國大陸市場吧。不管怎麼說,各位用戶現在能做的,就是在 2013 年 8 月 19 日前按照上圖中所提供的方法為自己的電郵搬家了。就通知來看,中國大陸雅虎官方給出的搬遷建議顯然是與他們有著密切關係的阿里雲(大陸雅虎的背後就是阿里巴巴啊)。當然,如果你有其它方向的話也可以透過雅虎提供的 POP 服務來完成遷移。總之無論你選擇哪一種方法,儘快讓自己與聯絡人熟悉起「新家」總是沒有錯的啦。最後,也對中國大陸雅虎郵箱道一聲珍重,畢竟對許多大陸用戶來說,這可是他們電郵的「第一次」啊。

shared via http://feedly.com

Google Glass gets unboxed

Wow... 原來Google Glass還附送太陽眼鏡啊... 那我不買太陽眼鏡是正確的...

 
 

Sent to you by Mac via Google Reader:

 
 

via Android and Me by Taylor Wimberly on 4/17/13

After a year of waiting, developers and early adopters have started to receive the Google Glass "Explorer Edition" units that were pre-ordered at Google IO 2012. The retail version of Google Glass is expected to be available later this year (for an unknown price that is lower than the $1500 Explorer Edition), but you can get a quick preview of what to expect thanks to a couple of unboxing videos that just hit the web.

Google IO 2013 is just around the corner, so we should get to spend some hands-on (or heads-on?) time with this exciting piece of technology soon enough. Developers who are interested in creating apps for Glass should check out the Quick Start Guide for the official API documentation.

If Google Glass turns out to be affordable (think price of a smartphone), how many of you will be interested in picking a pair up?


 
 

Things you can do from here:

 
 

Facebook 的 Memcache 架構…

Facebook 的 Memcache 架構…
http://feedly.com/k/12kQVTQ

NSDI '13Facebook 的工程師有講到 Facebook 內的 Memcache 的架構:「Scaling Memcache at Facebook」,有影片可以看,也有 PDF 投影片可以下載。

其實 2013 年這次的 conference 提到的架構以前就有提過了… 雖然一時間找不到之前的投影片,但還是可以配合著以前的文章與投影片看出 Facebook 怎麼利用 Memcache 架構 cache layer:

2007 年的「Largest production memcached install?」 2008 年的「Scaling Out」。 2008 年的「Caching Performance Lessons From Facebook」。 2008 年的「Scaling memcached at Facebook」。 2010 年在 QCon 的「memcache@facebook」 (PDF)。 2013 年的「McDipper: A key-value cache for Flash storage」。

Facebook 會這樣設計 Memcache 架構,跟 Facebook 用 PHP 的方式有關,是在 PHP 的限制下想辦法爭取效率的作法。

不過這些投影片裡的資料畢竟是有年代了,現在的 memcached 改善了很多,跟當年的情況不太一樣,看之前的投影片最好知道當時 memcached 有哪些問題會比較能理解 Facebook 的工程師們想要解決什麼問題。

Related Posts:

Facebook HipHop 的效率 Facebook 將全面支援 SPDY… W3 Total Cache 持續爛掉中… W3 Total Cache 0.9.2.6 的災情慘重… Google 對字串處理的最佳化

shared via http://feedly.com

傳聞 Windows 8.1 會重新加入「開始鍵」和啟動後直入桌面

傳聞 Windows 8.1 會重新加入「開始鍵」和啟動後直入桌面
http://feedly.com/k/15frBRV

或許 Microsoft 真的發現有不少傳統的 Windows 用戶都對 Windows 8 的平板界面和失落了的「開始鍵」沒有多少好感;看到 ModernMixStart8 的大賣就知道了;所以似乎 Microsoft 正在努力補回一切。Microsoft Portal 就在流出的 Windows Blue(Windows 8.1)源始碼中發現 TwinUI.dll 檔案裡寫著「CanSuppressStartScreen」的項目,暗示有可能代表 Windows 8.1 啟動時可以直接主入傳統的桌面頁,蓋過現有 Windows 8 的開始頁。另外,ZDNet 的 Mary Jo Foley 就從線人得知 Windows 8.1 的改變不只如此,還可能將以往在左下角的「開始鍵」重現在大家的螢幕上,並可設定其開關,不過她亦不忘補充說 Microsoft 只是「正在考慮」,一切仍然存在變數。

經由:Engadget
引用來源:ZDNetMicrosoft Portal

shared via http://feedly.com

星期二, 4月 16, 2013

Motorola’s “just right” size devices reported to run stock Android

Motorola's "just right" size devices reported to run stock Android
http://feedly.com/k/ZypwZr

The first batch of devices to come from the Motorola and Google partnership will run stock Android and feature the right size of displays. Using the mindset that "better is better" and not "bigger is better", these smartphones are reported to feature thin bezels not unlike the Motorola Razr M.

According to a recent PC Mag report, Motorola indicated that fans of stock Android and more practical sized devices will be pleased. Expected to arrive in the second half of the year these phones may ultimately fall under one line of branding; less Droid, Atrix, and Photon models to keep up with.

From a software and UI perspective, our strategy is to embrace Android and to make it the best expression of Android and Google in the market. It will be the unadulterated version of Android, and I feel really good about our embracing Android and being the best Android experience.


Perhaps more important than size, these new Motorola devices are expected to feature a stock version of Android with relatively little bloatware. Presumably the smartphones sold through Google's direct channel will be void of any added software while the versions sold through carriers might feature branded apps.

In terms of software experience we are still anticipating the "wow" factor that provides users with more than what happens with today's devices.

PC Mag

The post Motorola's "just right" size devices reported to run stock Android appeared first on AndroidGuys.

shared via http://feedly.com

Update on Tablet App Guidelines and Screenshots

Update on Tablet App Guidelines and Screenshots
http://feedly.com/k/ZvsUFy

Posted by Ellie Powers, Google Play team

More and more, developers are investing in a full tablet experience for their apps and seeing those investments pay off big. Starting today, it's even easier to show off those tablet apps to users and to understand what it takes to make a great tablet app.

Many of you have already made optimizations to your apps to make them more beautiful and useful on tablets, and we're giving you a way to showcase your application's specialized tablet layouts. You're now able to upload screenshots of your app running on 7" and 10" tablets to the Google Play Developer Console, and those screenshots are shown preferentially in Google Play to users on those devices.

In October, we published the tablet app quality checklist, which shares best practices for creating tablet applications that users will love. We've updated this checklist with additional tips and guidelines, as well as more details on specific technical checks you should perform to ensure your app is correctly optimized for tablets. Soon, we'll start surfacing tips based on our tablet app quality guidelines directly in the Google Play Developer Console, and we'll use these guidelines to help users better discover tablet apps in Google Play.

So upload your tablet screenshots and confirm your app follows the quality guidelines today.

Join the discussion on

+Android Developers

shared via http://feedly.com