讓 django 專案直接開上gae@coscup2011
DESCRIPTION
InTRANSCRIPT
![Page 2: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/2.jpg)
2011 年 2 月
![Page 3: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/3.jpg)
四個年輕人因為創業的夢
![Page 4: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/4.jpg)
他們組成了驚奇四超人?
![Page 5: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/5.jpg)
資訊技術
理財知識
![Page 6: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/6.jpg)
精實創業沒錢、沒閒、沒資源
![Page 7: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/7.jpg)
豐富の IT 資源?
Server ?Framework ?
頻寬?
效能?
NO WAY !!
![Page 8: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/8.jpg)
沒錢Prototype 還是要做!
![Page 9: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/9.jpg)
好在我們有廣大的免費 / OSS 資源
![Page 10: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/10.jpg)
找齊夥伴,朝偉大航道邁進,前進!!!
python
Django
GAE
![Page 11: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/11.jpg)
屎了!
![Page 12: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/12.jpg)
GAE 太害羞,不太想讓 DJANGO 騎 ^H 上毛病一堆
GAE
![Page 13: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/13.jpg)
NO, NO, NO~ 那一點都不 FASHION
I Don’t Want !• To Use google.appengine objs
• To Use only goolge user
management system
• The hassles of rewriting the
codes to port the legacy system
![Page 14: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/14.jpg)
“ 念”你我的字典裡沒有~放棄因為已鎖定 D
我從不做空白的 Coding ~因為已決定 D
眼見這月亮悄悄溜走又見那太陽緩緩升起多少 Solution 變換多少 Study Cases
Debug 不偷懶始終如一…
![Page 15: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/15.jpg)
終於!我們找到了… .
![Page 16: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/16.jpg)
一個框架與平台合作無間的故事
DJANGO NONREL
DJANGOAPPENGINE
![Page 17: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/17.jpg)
Django Nonrel
• 獨立於官方 Django 之外
• 加入 NoSQL Database 功能來支援 ORM
整個場面我 hold 住!Django Nonrel 表示:
![Page 18: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/18.jpg)
Django Nonrel 原理?
• Developers 對 Model 操作
• Django ORM
• SQL
• Django-dbindexer
• No-Sql DB Calls
![Page 19: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/19.jpg)
為何要選 Django Nonrel ?
• 可以把程式轉到 non-relational DBs 上
• 可以把程式 PO 回 SQL DBs 上
• 不用手動 / 程式處理 Denormalization
![Page 20: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/20.jpg)
為何要選 Django Nonrel ?
• 不用手動改寫下列 querie
( JOINs, select_related(), etc. ) < 並非全部支援
>
• 不會被鎖在單一平台提供商( GAE, etc. )
![Page 21: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/21.jpg)
Django Nonrel 目前支援
• GAE 跟 djangoappengine 混搭
• MongoDB 跟 MongoDB backend 混搭
![Page 22: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/22.jpg)
Django Nonrel 開發支援中的• ElasticSearch
• Cassandra
• 更多訊息:http://www.allbuttonspressed.com/projects/django-nonrel
![Page 23: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/23.jpg)
Djangoappengine
• 內藏 Django App Engine 後台整合:– DB
• 一定要跟 Django Nonrel 混搭
![Page 24: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/24.jpg)
來玩一下吧!
![Page 25: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/25.jpg)
從頭開始專案,吃蛋糕一樣 !
• 下載樣板專案• 複製所有 .py 與 .yaml 到你專案的根
目錄
• 接著:
![Page 26: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/26.jpg)
從頭開始專案,吃蛋糕一樣 !下載並複製以下資料夾到指定地點:
From To
django-nonrel / django <project> / django
djangotoolbox / djangotoolbox <project> / djangotoolbox
django-autoload / autoload <project> / autoload
django-dbindexer / dbindexer <project> / dbindexer
djangoappengine <project> / djangoappengine
![Page 27: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/27.jpg)
從頭開始專案,吃蛋糕一樣 !
目前專案的資料夾看起來是這樣:
<project> / django
<project> / djangotoolbox
<project> / autoload
<project> / dbindexer
<project> / djangoappengine
![Page 28: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/28.jpg)
那我的 Dependency 呢 ?
• 但是可以『打包』整包的原始碼呀 ~
• 放在專案根目錄下就 ok 啦!
耶?好像沒有辦法裝在 GAE 上 !?
![Page 29: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/29.jpg)
那我的 Dependency 呢 ?
• Zip 起來,放在根目錄的 zip-packages下
注意:請不要 zip 所有在 settings.py 中INSTALLED_APPS 下的 dependency
檔案太多了?GAE 有 3000 個檔案的限制!
![Page 30: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/30.jpg)
在自己的機器上試試看 !winston_awesome> python manage.py runserver
Try it!
![Page 31: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/31.jpg)
丟上 GAE 跑跑看吧!winston_awesome> python manage.py
deploy
Try it!
![Page 32: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/32.jpg)
DEMO
![Page 33: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/33.jpg)
It works!
![Page 34: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/34.jpg)
關於備份資料…
• 直接使用 Google 提供的工具!– Bulkloader
http://code.google.com/appengine/docs/pyth
on/tools/uploadingdata.html
• 如果你想用聽起來更屌的:– Gaebar
http://aralbalkan.com/1784(沒測過,不負責推薦)
![Page 35: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/35.jpg)
限制是…• 請不要用 JOIN !!
(沒差,對我來說, JOIN 很不直覺)
• 沒有 many-to-many 的關係• 沒有 Aggregates• 沒有 transactions
但可以用 GAE 的 run_in_transaction()
• 沒有 QuerySet.select_related()
![Page 36: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/36.jpg)
如果我想 PO 舊的 APP 呢 ?
• 改一下 setting.py 資料庫與 INSTALLED_APPS
• 把你 Dependency 的原始碼打包進來
• 對!這樣就 ok 了!
![Page 37: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/37.jpg)
DEMO
![Page 38: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/38.jpg)
GAE
開上?騎上?坐上?都行啦!反正…你懂得 >///<
![Page 39: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/39.jpg)
還有一些東西要注意!
• 如果你的 APP 有用 JOINs 或是其他沒有支援的語法
• 你必須要改寫過!
![Page 40: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/40.jpg)
把你 SQL 的資料放上 GAE ?
• 這是個『異常痛苦』的經驗!
• 試過很多辦法,包括混搭 django
terminal 與threads
![Page 41: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/41.jpg)
把你 SQL 的資料放上 GAE ?
如果你真的要做,請愛用 Google 的方式
google map reduce 與 CSV file 上傳 !
(但是!還是很難用!)
![Page 42: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/42.jpg)
用 CSV 上傳整個資料庫• 單一檔案大小上限: 1 MB
• 也就是說… . 如果你的資料庫 CSV 有 1G ,
你要上傳 1024 個檔案… XD
• 但這比較是 GAE 的問題啦!
![Page 43: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/43.jpg)
GAE 還有什麼問題?
免費的總是最貴,資源限制一堆
尤其是:
流量 / 頻寬!
![Page 44: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/44.jpg)
Yoooooooooo 它就漫出來了 oooooooo!!!
一不小心,就會發生杯具…
Pingooo 當司機
Pingooo 回到電腦前
(3616 hits, 295 IPs) @ 20:00
![Page 45: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/45.jpg)
後來,我總算學會了如何去省……
節省資源,從小地方做起!
![Page 46: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/46.jpg)
把靜態檔案切割搬出去!
![Page 47: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/47.jpg)
平均五次 / 單位 ms Yahoo GAE Google Code
Yahoo Logo( 2,338 Bytes )
With Cache 26.6 319.6 23
Without Cache 43 247.6 267
With Cache Waiting for Response 15.6 205.4 20.8
Receiving Data 0 0 0
Without Cache Waiting for Response 22.6 234.6 290.8
Receiving Data 13.2 0.8 1
Yahoo~~
![Page 48: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/48.jpg)
平均五次 / 單位 ms Pixnet GAE Google Code
Pixnet Header( 42,616 Bytes )
With Cache 26.6 244.4 21
Without Cache 81 788.2 524.6
With Cache Waiting for Response 17.2 180.6 18.8
Receiving Data 22.8 0 0
Without Cache Waiting for Response 50.2 206.6 266.8
Receiving Data 28.8 502.6 184.4
Pinext !
![Page 49: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/49.jpg)
平均五次 / 單位 ms MurMur GAE Google Code
MurMur Header( 34,775 Bytes )
With Cache 54.6 245.8 21.6
Without Cache 57.4 492.2 417
With Cache Waiting for Response 16 187 19.2
Receiving Data 0 0 0
Without Cache Waiting for Response 23.6 187.4 315.4
Receiving Data 23.2 486 147.8
Murmur….
![Page 50: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/50.jpg)
平均五次 / 單位 ms GoogleAPIs GAE Google Code
jQuery-1.4.1.min.js( 70,843 Bytes ) With Cache 22 249 23.4
Without Cache 45.4 524.6 452
With Cache Waiting for Response 17 189 19
Receiving Data 0 0 0
Without Cache Waiting for Response 20.6 192.2 290.8
Receiving Data 17 323.6 242
jQuery
![Page 51: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/51.jpg)
平均五次 / 單位 ms GAE Google Code
Hinet_10m.zip( 10,503,578 Bytes )
In GAE, the max size of a single file is 10485760 bytes, and the size of test file is 10503578 bytes.
519.8 KB/s
test.zip( 6,736,342 Bytes )
710 KB/s 305.2 KB/s
隨便包個測測…
![Page 52: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/52.jpg)
Bonus Session
• Cache
– 直接使用 Django 對 memcache
module 的 API ,不需要使用 GAE 提供的
– memcache backend 預設是開啟的
![Page 53: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/53.jpg)
Bonus Session
• 檔案上傳 / 下載
– 請愛用 django-filetransfers
• 複製“ filetransfers” 資料夾到根目錄
• 新增“ filetransfers” 到
INSTALLED_APPS
![Page 54: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/54.jpg)
Bonus Session
• 檔案上傳 / 下載
– App Engine Blobstore 預設是開啟的
– 詳細資料http
://www.allbuttonspressed.com/projects/django
-filetransfers
![Page 55: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/55.jpg)
Bonus Session
• Task– 還沒好
DjangoAppEngine 團隊正在開發中… .
– 這時候,只能用 Google 的http://code.google.com/appengine/docs/pytho
n/taskqueue/
![Page 56: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/56.jpg)
To GAE or Not To GAE?
• Task• Transaction
![Page 57: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/57.jpg)
Djnago Nonrel + Djangoappengine
1. django-social-auth
2. 有個舊的 django app
結論 – 何時用?
![Page 58: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/58.jpg)
• 請記住:不要用 JOINs (或其他不支援的語法)
結論 – 切忌
![Page 59: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/59.jpg)
Django Nonrel + Djangoappengine 最好拿來
1. Prototyping
2. Homework
結論 – 最適合
![Page 60: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/60.jpg)
• 誰知道?
• 說不定你會愛上 GAE 上的 Django
結論 – 所以到底要不要?
![Page 61: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/61.jpg)
參考資料• Django-nonrel & webdev blog
– http://www.allbuttonspressed.com/
• Django Snippets– http://djangosnippets.org/snippets/1003/
• Django– https://www.djangoproject.com/
• Google App Engine– http://code.google.com/appengine/
![Page 62: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/62.jpg)
更多後續
• 好險有科技
http://techblog.insureme.com.tw
• 好險網
http://www.insureme.com.tw
![Page 63: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/63.jpg)
感謝
• 好險技術團隊所有夥伴
• Eric Wu, EragonJ, ygtw, 藏鏡人一號
• COSCUP 2011 籌備團隊
• 各位台下聆聽的你!
陳翅膀、王貓貓,下台鞠躬
![Page 64: 讓 Django 專案直接開上GAE@coscup2011](https://reader034.vdocuments.site/reader034/viewer/2022042513/54c650394a7959ad7b8b45da/html5/thumbnails/64.jpg)
Q & A ?