Merge branch 'master' into patch-1
Oleksandr Kisilenko
GitHub
commit
de616fb3c6
|
|
@ -1,4 +1,5 @@
|
|||
# Byte-compiled / optimized / DLL files
|
||||
*.epub
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
|
||||
|
|
|
|||
68
README-ja.md
68
README-ja.md
|
|
@ -1,9 +1,9 @@
|
|||
*[English](README.md) ∙ [简体中文](README-zh-Hans.md) | [Brazilian Portuguese](https://github.com/donnemartin/system-design-primer/issues/40) ∙ [Polish](https://github.com/donnemartin/system-design-primer/issues/68) ∙ [Russian](https://github.com/donnemartin/system-design-primer/issues/87) ∙ [Turkish](https://github.com/donnemartin/system-design-primer/issues/39) | [Add Translation](https://github.com/donnemartin/system-design-primer/issues/28)*
|
||||
*[English](README.md) ∙ [日本語](README-ja.md) ∙ [简体中文](README-zh-Hans.md) ∙ [繁體中文](README-zh-TW.md) | [العَرَبِيَّة](https://github.com/donnemartin/system-design-primer/issues/170) ∙ [বাংলা](https://github.com/donnemartin/system-design-primer/issues/220) ∙ [Português do Brasil](https://github.com/donnemartin/system-design-primer/issues/40) ∙ [Deutsch](https://github.com/donnemartin/system-design-primer/issues/186) ∙ [ελληνικά](https://github.com/donnemartin/system-design-primer/issues/130) ∙ [עברית](https://github.com/donnemartin/system-design-primer/issues/272) ∙ [Italiano](https://github.com/donnemartin/system-design-primer/issues/104) ∙ [韓國語](https://github.com/donnemartin/system-design-primer/issues/102) ∙ [فارسی](https://github.com/donnemartin/system-design-primer/issues/110) ∙ [Polski](https://github.com/donnemartin/system-design-primer/issues/68) ∙ [русский язык](https://github.com/donnemartin/system-design-primer/issues/87) ∙ [Español](https://github.com/donnemartin/system-design-primer/issues/136) ∙ [ภาษาไทย](https://github.com/donnemartin/system-design-primer/issues/187) ∙ [Türkçe](https://github.com/donnemartin/system-design-primer/issues/39) ∙ [tiếng Việt](https://github.com/donnemartin/system-design-primer/issues/127) ∙ [Français](https://github.com/donnemartin/system-design-primer/issues/250) | [Add Translation](https://github.com/donnemartin/system-design-primer/issues/28)*
|
||||
|
||||
# システム設計入門
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/jj3A5N8.png">
|
||||
<img src="http://i.imgur.com/jj3A5N8.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -44,7 +44,7 @@
|
|||
## 暗記カード
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/zdCAkB3.png">
|
||||
<img src="http://i.imgur.com/zdCAkB3.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -61,7 +61,7 @@
|
|||
コード技術面接用の問題を探している場合は[**こちら**](https://github.com/donnemartin/interactive-coding-challenges)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/b4YtAEN.png">
|
||||
<img src="http://i.imgur.com/b4YtAEN.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -91,7 +91,7 @@
|
|||
> それぞれのセクションはより学びを深めるような他の文献へのリンクが貼られています。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/jrUBAF7.png">
|
||||
<img src="http://i.imgur.com/jrUBAF7.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -436,7 +436,7 @@
|
|||
### CAP 理論
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/bgLMI2u.png">
|
||||
<img src="http://i.imgur.com/bgLMI2u.png"/>
|
||||
<br/>
|
||||
<i><a href=http://robertgreiner.com/2014/08/cap-theorem-revisited>Source: CAP theorem revisited</a></i>
|
||||
</p>
|
||||
|
|
@ -530,7 +530,7 @@
|
|||
## ドメインネームシステム
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/IOyLj4i.jpg">
|
||||
<img src="http://i.imgur.com/IOyLj4i.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/srikrupa5/dns-security-presentation-issa>Source: DNS security presentation</a></i>
|
||||
</p>
|
||||
|
|
@ -568,7 +568,7 @@ DNSは少数のオーソライズされたサーバーが上位に位置する
|
|||
## コンテンツデリバリーネットワーク(Content delivery network)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/h9TAuGI.jpg">
|
||||
<img src="http://i.imgur.com/h9TAuGI.jpg"/>
|
||||
<br/>
|
||||
<i><a href=https://www.creative-artworks.eu/why-use-a-content-delivery-network-cdn/>Source: Why use a CDN</a></i>
|
||||
</p>
|
||||
|
|
@ -609,7 +609,7 @@ CDNを用いてコンテンツを配信することで以下の二つの理由
|
|||
## ロードバランサー
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/h81n9iK.png">
|
||||
<img src="http://i.imgur.com/h81n9iK.png"/>
|
||||
<br/>
|
||||
<i><a href=http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html>Source: Scalable system design patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -679,7 +679,7 @@ Layer 7 ロードバランサーは [アプリケーションレイヤー](#通
|
|||
## リバースプロキシ(webサーバー)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/n41Azff.png">
|
||||
<img src="http://i.imgur.com/n41Azff.png"/>
|
||||
<br/>
|
||||
<i><a href=https://upload.wikimedia.org/wikipedia/commons/6/67/Reverse_proxy_h2g2bob.svg>Source: Wikipedia</a></i>
|
||||
<br/>
|
||||
|
|
@ -722,7 +722,7 @@ Layer 7 ロードバランサーは [アプリケーションレイヤー](#通
|
|||
## アプリケーション層
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/yB5SYwm.png">
|
||||
<img src="http://i.imgur.com/yB5SYwm.png"/>
|
||||
<br/>
|
||||
<i><a href=http://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer>Source: Intro to architecting systems for scale</a></i>
|
||||
</p>
|
||||
|
|
@ -759,7 +759,7 @@ Layer 7 ロードバランサーは [アプリケーションレイヤー](#通
|
|||
## データベース
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/Xkm5CXz.png">
|
||||
<img src="http://i.imgur.com/Xkm5CXz.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.youtube.com/watch?v=w95murBkYmU>Source: Scaling up to your first 10 million users</a></i>
|
||||
</p>
|
||||
|
|
@ -782,7 +782,7 @@ SQLなどのリレーショナルデータベースはテーブルに整理さ
|
|||
マスターデータベースが読み取りと書き込みを処理し、書き込みを一つ以上のスレーブデータベースに複製します。スレーブデータベースは読み取りのみを処理します。スレーブデータベースは木構造のように追加のスレーブにデータを複製することもできます。マスターデータベースがオフラインになった場合には、いずれかのスレーブがマスターに昇格するか、新しいマスターデータベースが追加されるまでは読み取り専用モードで稼働します。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/C9ioGtn.png">
|
||||
<img src="http://i.imgur.com/C9ioGtn.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -797,7 +797,7 @@ SQLなどのリレーショナルデータベースはテーブルに整理さ
|
|||
いずれのマスターも読み取り書き込みの両方に対応する。書き込みに関してはそれぞれ協調する。いずれかのマスターが落ちても、システム全体としては読み書き両方に対応したまま運用できる。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/krAHLGg.png">
|
||||
<img src="http://i.imgur.com/krAHLGg.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -825,7 +825,7 @@ SQLなどのリレーショナルデータベースはテーブルに整理さ
|
|||
#### Federation
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/U3qV33e.png">
|
||||
<img src="http://i.imgur.com/U3qV33e.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.youtube.com/watch?v=w95murBkYmU>Source: Scaling up to your first 10 million users</a></i>
|
||||
</p>
|
||||
|
|
@ -846,7 +846,7 @@ SQLなどのリレーショナルデータベースはテーブルに整理さ
|
|||
#### シャーディング
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/wU8x5Id.png">
|
||||
<img src="http://i.imgur.com/wU8x5Id.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -955,7 +955,7 @@ NoSQL は **key-value store**、 **document-store**、 **wide column store**、
|
|||
|
||||
> 概要: ハッシュテーブル
|
||||
|
||||
キーバリューストアでは一般的に0、1の読み、書きができ、それらはメモリないしSSDで裏付けられています。データストアはキーを [辞書的順序](https://en.wikipedia.org/wiki/Lexicographical_order) で保持することでキーの効率的な取得を可能にしています。キーバリューストアではメタデータを値とともに保持することが可能です。
|
||||
キーバリューストアでは一般的にO(1)の読み書きができ、それらはメモリないしSSDで裏付けられています。データストアはキーを [辞書的順序](https://en.wikipedia.org/wiki/Lexicographical_order) で保持することでキーの効率的な取得を可能にしています。キーバリューストアではメタデータを値とともに保持することが可能です。
|
||||
|
||||
キーバリューストアはハイパフォーマンスな挙動が可能で、単純なデータモデルやインメモリーキャッシュレイヤーなどのデータが急速に変わる場合などに使われます。単純な処理のみに機能が制限されているので、追加の処理機能が必要な場合にはその複雑性はアプリケーション層に載せることになります。
|
||||
|
||||
|
|
@ -990,7 +990,7 @@ NoSQL は **key-value store**、 **document-store**、 **wide column store**、
|
|||
#### ワイドカラムストア
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/n16iOGk.png">
|
||||
<img src="http://i.imgur.com/n16iOGk.png"/>
|
||||
<br/>
|
||||
<i><a href=http://blog.grio.com/2015/11/sql-nosql-a-brief-history.html>Source: SQL & NoSQL, a brief history</a></i>
|
||||
</p>
|
||||
|
|
@ -1013,7 +1013,7 @@ Googleは[Bigtable](http://www.read.seas.harvard.edu/~kohler/class/cs239-w08/cha
|
|||
#### グラフデータベース
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/fNcl65g.png">
|
||||
<img src="http://i.imgur.com/fNcl65g.png"/>
|
||||
<br/>
|
||||
<i><a href=https://en.wikipedia.org/wiki/File:GraphDatabase_PropertyGraph.png>Source: Graph database</a></i>
|
||||
</p>
|
||||
|
|
@ -1041,7 +1041,7 @@ Googleは[Bigtable](http://www.read.seas.harvard.edu/~kohler/class/cs239-w08/cha
|
|||
### SQLか？NoSQLか？
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/wXGqG5f.png">
|
||||
<img src="http://i.imgur.com/wXGqG5f.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.infoq.com/articles/Transition-RDBMS-NoSQL/>Source: Transitioning from RDBMS to NoSQL</a></i>
|
||||
</p>
|
||||
|
|
@ -1083,7 +1083,7 @@ NoSQLに適するサンプルデータ:
|
|||
## キャッシュ
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/Q6z24La.png">
|
||||
<img src="http://i.imgur.com/Q6z24La.png"/>
|
||||
<br/>
|
||||
<i><a href=http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html>Source: Scalable system design patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -1154,7 +1154,7 @@ Redisはさらに以下のような機能を備えています:
|
|||
#### キャッシュアサイド
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/ONjORqk.png">
|
||||
<img src="http://i.imgur.com/ONjORqk.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast>Source: From cache to in-memory data grid</a></i>
|
||||
</p>
|
||||
|
|
@ -1166,7 +1166,7 @@ Redisはさらに以下のような機能を備えています:
|
|||
* エントリをキャッシュに追加します
|
||||
* エントリを返します
|
||||
|
||||
```
|
||||
```python
|
||||
def get_user(self, user_id):
|
||||
user = cache.get("user.{0}", user_id)
|
||||
if user is None:
|
||||
|
|
@ -1190,7 +1190,7 @@ def get_user(self, user_id):
|
|||
#### ライトスルー
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/0vBc0hN.png">
|
||||
<img src="http://i.imgur.com/0vBc0hN.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -1209,7 +1209,7 @@ set_user(12345, {"foo":"bar"})
|
|||
|
||||
キャッシュコード:
|
||||
|
||||
```
|
||||
```python
|
||||
def set_user(user_id, values):
|
||||
user = db.query("UPDATE Users WHERE id = {0}", user_id, values)
|
||||
cache.set(user_id, user)
|
||||
|
|
@ -1225,7 +1225,7 @@ def set_user(user_id, values):
|
|||
#### ライトビハインド (ライトバック)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/rgSrvjG.png">
|
||||
<img src="http://i.imgur.com/rgSrvjG.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -1243,7 +1243,7 @@ def set_user(user_id, values):
|
|||
#### リフレッシュアヘッド
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/kxtjqgE.png">
|
||||
<img src="http://i.imgur.com/kxtjqgE.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast>Source: From cache to in-memory data grid</a></i>
|
||||
</p>
|
||||
|
|
@ -1275,7 +1275,7 @@ def set_user(user_id, values):
|
|||
## 非同期処理
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/54GYsSx.png">
|
||||
<img src="http://i.imgur.com/54GYsSx.png"/>
|
||||
<br/>
|
||||
<i><a href=http://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer>Source: Intro to architecting systems for scale</a></i>
|
||||
</p>
|
||||
|
|
@ -1321,7 +1321,7 @@ def set_user(user_id, values):
|
|||
## 通信
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/5KeocQs.jpg">
|
||||
<img src="http://i.imgur.com/5KeocQs.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.escotal.com/osilayer.html>Source: OSI 7 layer model</a></i>
|
||||
</p>
|
||||
|
|
@ -1353,7 +1353,7 @@ HTTPは**TCP** や **UDP** などの低級プロトコルに依存している
|
|||
### 伝送制御プロトコル (TCP)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/JdAsdvG.jpg">
|
||||
<img src="http://i.imgur.com/JdAsdvG.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.wildbunny.co.uk/blog/2012/10/09/how-to-make-a-multi-player-game-part-1/>Source: How to make a multiplayer game</a></i>
|
||||
</p>
|
||||
|
|
@ -1377,7 +1377,7 @@ TCPは高い依存性を要し、時間制約が厳しくないものに適し
|
|||
### ユーザデータグラムプロトコル (UDP)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/yzDrJtA.jpg">
|
||||
<img src="http://i.imgur.com/yzDrJtA.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.wildbunny.co.uk/blog/2012/10/09/how-to-make-a-multi-player-game-part-1/>Source: How to make a multiplayer game</a></i>
|
||||
</p>
|
||||
|
|
@ -1406,7 +1406,7 @@ TCPよりもUDPを使うのは:
|
|||
### 遠隔手続呼出 (RPC)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/iF4Mkb5.png">
|
||||
<img src="http://i.imgur.com/iF4Mkb5.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.puncsky.com/blog/2016-02-13-crack-the-system-design-interview>Source: Crack the system design interview</a></i>
|
||||
</p>
|
||||
|
|
@ -1602,7 +1602,7 @@ Notes
|
|||
| 質問 | 解答 |
|
||||
|---|---|
|
||||
| Dropboxのようなファイル同期サービスを設計する | [youtube.com](https://www.youtube.com/watch?v=PE4gwstWhmc) |
|
||||
| Googleのような検索エンジンの設計 | [queue.acm.org](http://queue.acm.org/detail.cfm?id=988407)<br/>[stackexchange.com](http://programmers.stackexchange.com/questions/38324/interview-question-how-would-you-implement-google-search)<br/>[ardendertat.com](http://www.ardendertat.com/2012/01/11/implementing-search-engines/)<br>[stanford.edu](http://infolab.stanford.edu/~backrub/google.html) |
|
||||
| Googleのような検索エンジンの設計 | [queue.acm.org](http://queue.acm.org/detail.cfm?id=988407)<br/>[stackexchange.com](http://programmers.stackexchange.com/questions/38324/interview-question-how-would-you-implement-google-search)<br/>[ardendertat.com](http://www.ardendertat.com/2012/01/11/implementing-search-engines/)<br/>[stanford.edu](http://infolab.stanford.edu/~backrub/google.html) |
|
||||
| Googleのようなスケーラブルなwebクローラーの設計 | [quora.com](https://www.quora.com/How-can-I-build-a-web-crawler-from-scratch) |
|
||||
| Google docsの設計 | [code.google.com](https://code.google.com/p/google-mobwrite/)<br/>[neil.fraser.name](https://neil.fraser.name/writing/sync/) |
|
||||
| Redisのようなキーバリューストアの設計 | [slideshare.net](http://www.slideshare.net/dvirsky/introduction-to-redis) |
|
||||
|
|
@ -1629,7 +1629,7 @@ Notes
|
|||
> 世の中のシステムがどのように設計されているかについての記事
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/TcUo2fw.png">
|
||||
<img src="http://i.imgur.com/TcUo2fw.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.infoq.com/presentations/Twitter-Timeline-Scalability>Source: Twitter timelines at scale</a></i>
|
||||
</p>
|
||||
|
|
|
|||
|
|
@ -3,10 +3,12 @@
|
|||
> * 译者：[XatMassacrE](https://github.com/XatMassacrE)、[L9m](https://github.com/L9m)、[Airmacho](https://github.com/Airmacho)、[xiaoyusilen](https://github.com/xiaoyusilen)、[jifaxu](https://github.com/jifaxu)
|
||||
> * 这个 [链接](https://github.com/xitu/system-design-primer/compare/master...donnemartin:master) 用来查看本翻译与英文版是否有差别（如果你没有看到 README.md 发生变化，那就意味着这份翻译文档是最新的）。
|
||||
|
||||
*[English](README.md) ∙ [日本語](README-ja.md) ∙ [简体中文](README-zh-Hans.md) ∙ [繁體中文](README-zh-TW.md) | [العَرَبِيَّة](https://github.com/donnemartin/system-design-primer/issues/170) ∙ [বাংলা](https://github.com/donnemartin/system-design-primer/issues/220) ∙ [Português do Brasil](https://github.com/donnemartin/system-design-primer/issues/40) ∙ [Deutsch](https://github.com/donnemartin/system-design-primer/issues/186) ∙ [ελληνικά](https://github.com/donnemartin/system-design-primer/issues/130) ∙ [עברית](https://github.com/donnemartin/system-design-primer/issues/272) ∙ [Italiano](https://github.com/donnemartin/system-design-primer/issues/104) ∙ [韓國語](https://github.com/donnemartin/system-design-primer/issues/102) ∙ [فارسی](https://github.com/donnemartin/system-design-primer/issues/110) ∙ [Polski](https://github.com/donnemartin/system-design-primer/issues/68) ∙ [русский язык](https://github.com/donnemartin/system-design-primer/issues/87) ∙ [Español](https://github.com/donnemartin/system-design-primer/issues/136) ∙ [ภาษาไทย](https://github.com/donnemartin/system-design-primer/issues/187) ∙ [Türkçe](https://github.com/donnemartin/system-design-primer/issues/39) ∙ [tiếng Việt](https://github.com/donnemartin/system-design-primer/issues/127) ∙ [Français](https://github.com/donnemartin/system-design-primer/issues/250) | [Add Translation](https://github.com/donnemartin/system-design-primer/issues/28)*
|
||||
|
||||
# 系统设计入门
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/jj3A5N8.png">
|
||||
<img src="http://i.imgur.com/jj3A5N8.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -55,7 +57,7 @@
|
|||
## 抽认卡
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/zdCAkB3.png">
|
||||
<img src="http://i.imgur.com/zdCAkB3.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -72,7 +74,7 @@
|
|||
你正在寻找资源以准备[**编程面试**](https://github.com/donnemartin/interactive-coding-challenges)吗？
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/b4YtAEN.png">
|
||||
<img src="http://i.imgur.com/b4YtAEN.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -102,7 +104,7 @@
|
|||
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/jrUBAF7.png">
|
||||
<img src="http://i.imgur.com/jrUBAF7.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -298,7 +300,7 @@
|
|||
|
||||
| 问题 | |
|
||||
| ---------------------------------------- | ---------------------------------------- |
|
||||
| 设计 Pastebin.com (或者 Bit.ly) | [解答](solutions/system_design/pastebin/README.md) |
|
||||
| 设计 Pastebin.com (或者 Bit.ly) | [解答](solutions/system_design/pastebin/README-zh-Hans.md) |
|
||||
| 设计 Twitter 时间线和搜索 (或者 Facebook feed 和搜索) | [解答](solutions/system_design/twitter/README.md) |
|
||||
| 设计一个网页爬虫 | [解答](solutions/system_design/web_crawler/README.md) |
|
||||
| 设计 Mint.com | [解答](solutions/system_design/mint/README.md) |
|
||||
|
|
@ -446,7 +448,7 @@
|
|||
### CAP 理论
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/bgLMI2u.png">
|
||||
<img src="http://i.imgur.com/bgLMI2u.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://robertgreiner.com/2014/08/cap-theorem-revisited">来源：再看 CAP 理论</a></strong>
|
||||
</p>
|
||||
|
|
@ -541,7 +543,7 @@ DNS 和 email 等系统使用的是此种方式。最终一致性在高可用性
|
|||
## 域名系统
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/IOyLj4i.jpg">
|
||||
<img src="http://i.imgur.com/IOyLj4i.jpg"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.slideshare.net/srikrupa5/dns-security-presentation-issa">来源：DNS 安全介绍</a></strong>
|
||||
</p>
|
||||
|
|
@ -579,7 +581,7 @@ DNS 和 email 等系统使用的是此种方式。最终一致性在高可用性
|
|||
## 内容分发网络（CDN）
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/h9TAuGI.jpg">
|
||||
<img src="http://i.imgur.com/h9TAuGI.jpg"/>
|
||||
<br/>
|
||||
<strong><a href="https://www.creative-artworks.eu/why-use-a-content-delivery-network-cdn/">来源：为什么使用 CDN</a></strong>
|
||||
</p>
|
||||
|
|
@ -618,7 +620,7 @@ CDN 拉取是当第一个用户请求该资源时，从服务器上拉取资源
|
|||
## 负载均衡器
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/h81n9iK.png">
|
||||
<img src="http://i.imgur.com/h81n9iK.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html">来源：可扩展的系统设计模式</a></strong>
|
||||
</p>
|
||||
|
|
@ -687,7 +689,7 @@ CDN 拉取是当第一个用户请求该资源时，从服务器上拉取资源
|
|||
## 反向代理（web 服务器）
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/n41Azff.png">
|
||||
<img src="http://i.imgur.com/n41Azff.png"/>
|
||||
<br/>
|
||||
<strong><a href="https://upload.wikimedia.org/wikipedia/commons/6/67/Reverse_proxy_h2g2bob.svg">资料来源：维基百科</a></strong>
|
||||
<br/>
|
||||
|
|
@ -731,7 +733,7 @@ CDN 拉取是当第一个用户请求该资源时，从服务器上拉取资源
|
|||
## 应用层
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/yB5SYwm.png">
|
||||
<img src="http://i.imgur.com/yB5SYwm.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer">资料来源：可缩放系统构架介绍</a></strong>
|
||||
</p>
|
||||
|
|
@ -769,7 +771,7 @@ CDN 拉取是当第一个用户请求该资源时，从服务器上拉取资源
|
|||
## 数据库
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/Xkm5CXz.png">
|
||||
<img src="http://i.imgur.com/Xkm5CXz.png"/>
|
||||
<br/>
|
||||
<strong><a href="https://www.youtube.com/watch?v=w95murBkYmU">资料来源：扩展你的用户数到第一个一千万</a></strong>
|
||||
</p>
|
||||
|
|
@ -790,7 +792,7 @@ CDN 拉取是当第一个用户请求该资源时，从服务器上拉取资源
|
|||
关系型数据库扩展包括许多技术：**主从复制**、**主主复制**、**联合**、**分片**、**非规范化**和 **SQL调优**。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/C9ioGtn.png">
|
||||
<img src="http://i.imgur.com/C9ioGtn.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.slideshare.net/jboner/scalability-availability-stability-patterns/">资料来源：可扩展性、可用性、稳定性、模式</a></strong>
|
||||
</p>
|
||||
|
|
@ -805,7 +807,7 @@ CDN 拉取是当第一个用户请求该资源时，从服务器上拉取资源
|
|||
- 参考[不利之处：复制](#不利之处复制)中，主从复制和主主复制**共同**的问题。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/krAHLGg.png">
|
||||
<img src="http://i.imgur.com/krAHLGg.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.slideshare.net/jboner/scalability-availability-stability-patterns/">资料来源：可扩展性、可用性、稳定性、模式</a></strong>
|
||||
</p>
|
||||
|
|
@ -840,7 +842,7 @@ CDN 拉取是当第一个用户请求该资源时，从服务器上拉取资源
|
|||
#### 联合
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/U3qV33e.png">
|
||||
<img src="http://i.imgur.com/U3qV33e.png"/>
|
||||
<br/>
|
||||
<strong><a href="https://www.youtube.com/watch?v=w95murBkYmU">资料来源：扩展你的用户数到第一个一千万</a></strong>
|
||||
</p>
|
||||
|
|
@ -862,7 +864,7 @@ CDN 拉取是当第一个用户请求该资源时，从服务器上拉取资源
|
|||
#### 分片
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/wU8x5Id.png">
|
||||
<img src="http://i.imgur.com/wU8x5Id.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.slideshare.net/jboner/scalability-availability-stability-patterns/">资料来源：可扩展性、可用性、稳定性、模式</a></strong>
|
||||
</p>
|
||||
|
|
@ -1006,7 +1008,7 @@ MongoDB 和 CouchDB 等一些文档类型存储还提供了类似 SQL 语言的
|
|||
#### 列型存储
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/n16iOGk.png">
|
||||
<img src="http://i.imgur.com/n16iOGk.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://blog.grio.com/2015/11/sql-nosql-a-brief-history.html">资料来源: SQL 和 NoSQL，一个简短的历史</a></strong>
|
||||
</p>
|
||||
|
|
@ -1029,9 +1031,9 @@ Google 发布了第一个列型存储数据库 [Bigtable](http://www.read.seas.h
|
|||
#### 图数据库
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/fNcl65g.png">
|
||||
<img src="http://i.imgur.com/fNcl65g.png"/>
|
||||
<br/>
|
||||
<strong><a href="https://en.wikipedia.org/wiki/File:GraphDatabase_PropertyGraph.png">资料来源：图数据库</a></strong>
|
||||
<strong><a href="https://en.wikipedia.org/wiki/File:GraphDatabase_PropertyGraph.png"/>资料来源：图数据库</a></strong>
|
||||
</p>
|
||||
|
||||
> 抽象模型： 图
|
||||
|
|
@ -1056,7 +1058,7 @@ Google 发布了第一个列型存储数据库 [Bigtable](http://www.read.seas.h
|
|||
### SQL 还是 NoSQL
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/wXGqG5f.png">
|
||||
<img src="http://i.imgur.com/wXGqG5f.png"/>
|
||||
<br/>
|
||||
<strong><a href="https://www.infoq.com/articles/Transition-RDBMS-NoSQL/">资料来源：从 RDBMS 转换到 NoSQL</a></strong>
|
||||
</p>
|
||||
|
|
@ -1097,7 +1099,7 @@ Google 发布了第一个列型存储数据库 [Bigtable](http://www.read.seas.h
|
|||
## 缓存
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/Q6z24La.png">
|
||||
<img src="http://i.imgur.com/Q6z24La.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html">资料来源：可扩展的系统设计模式</a></strong>
|
||||
</p>
|
||||
|
|
@ -1168,7 +1170,7 @@ Redis 有下列附加功能：
|
|||
#### 缓存模式
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/ONjORqk.png">
|
||||
<img src="http://i.imgur.com/ONjORqk.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast">资料来源：从缓存到内存数据网格</a></strong>
|
||||
</p>
|
||||
|
|
@ -1180,7 +1182,7 @@ Redis 有下列附加功能：
|
|||
- 将查找到的结果存储到缓存中
|
||||
- 返回所需内容
|
||||
|
||||
```
|
||||
```python
|
||||
def get_user(self, user_id):
|
||||
user = cache.get("user.{0}", user_id)
|
||||
if user is None:
|
||||
|
|
@ -1204,7 +1206,7 @@ def get_user(self, user_id):
|
|||
#### 直写模式
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/0vBc0hN.png">
|
||||
<img src="http://i.imgur.com/0vBc0hN.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.slideshare.net/jboner/scalability-availability-stability-patterns/">资料来源：可扩展性、可用性、稳定性、模式</a></strong>
|
||||
</p>
|
||||
|
|
@ -1223,7 +1225,7 @@ set_user(12345, {"foo":"bar"})
|
|||
|
||||
缓存代码：
|
||||
|
||||
```
|
||||
```python
|
||||
def set_user(user_id, values):
|
||||
user = db.query("UPDATE Users WHERE id = {0}", user_id, values)
|
||||
cache.set(user_id, user)
|
||||
|
|
@ -1239,7 +1241,7 @@ def set_user(user_id, values):
|
|||
#### 回写模式
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/rgSrvjG.png">
|
||||
<img src="http://i.imgur.com/rgSrvjG.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.slideshare.net/jboner/scalability-availability-stability-patterns/">资料来源：可扩展性、可用性、稳定性、模式</a></strong>
|
||||
</p>
|
||||
|
|
@ -1257,7 +1259,7 @@ def set_user(user_id, values):
|
|||
#### 刷新
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/kxtjqgE.png">
|
||||
<img src="http://i.imgur.com/kxtjqgE.png"/>
|
||||
<br/>
|
||||
<strong><a href=http://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast>资料来源：从缓存到内存数据网格</a></strong>
|
||||
</p>
|
||||
|
|
@ -1289,7 +1291,7 @@ def set_user(user_id, values):
|
|||
## 异步
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/54GYsSx.png">
|
||||
<img src="http://i.imgur.com/54GYsSx.png"/>
|
||||
<br/>
|
||||
<strong><a href=http://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer>资料来源：可缩放系统构架介绍</a></strong>
|
||||
</p>
|
||||
|
|
@ -1335,7 +1337,7 @@ def set_user(user_id, values):
|
|||
## 通讯
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/5KeocQs.jpg">
|
||||
<img src="http://i.imgur.com/5KeocQs.jpg"/>
|
||||
<br/>
|
||||
<strong><a href=http://www.escotal.com/osilayer.html>资料来源：OSI 7层模型</a></strong>
|
||||
</p>
|
||||
|
|
@ -1370,7 +1372,7 @@ HTTP 是依赖于较低级协议（如 **TCP** 和 **UDP**）的应用层协议
|
|||
### 传输控制协议（TCP）
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/JdAsdvG.jpg">
|
||||
<img src="http://i.imgur.com/JdAsdvG.jpg"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.wildbunny.co.uk/blog/2012/10/09/how-to-make-a-multi-player-game-part-1/">资料来源：如何制作多人游戏</a></strong>
|
||||
</p>
|
||||
|
|
@ -1394,7 +1396,7 @@ TCP 对于需要高可靠性但时间紧迫的应用程序很有用。比如包
|
|||
### 用户数据报协议（UDP）
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/yzDrJtA.jpg">
|
||||
<img src="http://i.imgur.com/yzDrJtA.jpg"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.wildbunny.co.uk/blog/2012/10/09/how-to-make-a-multi-player-game-part-1">资料来源：如何制作多人游戏</a></strong>
|
||||
</p>
|
||||
|
|
@ -1423,7 +1425,7 @@ UDP 可靠性更低但适合用在网络电话、视频聊天，流媒体和实
|
|||
### 远程过程调用协议（RPC）
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/iF4Mkb5.png">
|
||||
<img src="http://i.imgur.com/iF4Mkb5.png"/>
|
||||
<br/>
|
||||
<strong><a href="http://www.puncsky.com/blog/2016/02/14/crack-the-system-design-interview">Source: Crack the system design interview</a></strong>
|
||||
</p>
|
||||
|
|
@ -1618,7 +1620,7 @@ Notes
|
|||
| 问题 | 引用 |
|
||||
| ----------------------- | ---------------------------------------- |
|
||||
| 设计类似于 Dropbox 的文件同步服务 | [youtube.com](https://www.youtube.com/watch?v=PE4gwstWhmc) |
|
||||
| 设计类似于 Google 的搜索引擎 | [queue.acm.org](http://queue.acm.org/detail.cfm?id=988407)<br/>[stackexchange.com](http://programmers.stackexchange.com/questions/38324/interview-question-how-would-you-implement-google-search)<br/>[ardendertat.com](http://www.ardendertat.com/2012/01/11/implementing-search-engines/)<br>[stanford.edu](http://infolab.stanford.edu/~backrub/google.html) |
|
||||
| 设计类似于 Google 的搜索引擎 | [queue.acm.org](http://queue.acm.org/detail.cfm?id=988407)<br/>[stackexchange.com](http://programmers.stackexchange.com/questions/38324/interview-question-how-would-you-implement-google-search)<br/>[ardendertat.com](http://www.ardendertat.com/2012/01/11/implementing-search-engines/)<br/>[stanford.edu](http://infolab.stanford.edu/~backrub/google.html) |
|
||||
| 设计类似于 Google 的可扩展网络爬虫 | [quora.com](https://www.quora.com/How-can-I-build-a-web-crawler-from-scratch) |
|
||||
| 设计 Google 文档 | [code.google.com](https://code.google.com/p/google-mobwrite/)<br/>[neil.fraser.name](https://neil.fraser.name/writing/sync/) |
|
||||
| 设计类似 Redis 的键值存储 | [slideshare.net](http://www.slideshare.net/dvirsky/introduction-to-redis) |
|
||||
|
|
@ -1645,7 +1647,7 @@ Notes
|
|||
> 关于现实中真实的系统是怎么设计的文章。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/TcUo2fw.png">
|
||||
<img src="http://i.imgur.com/TcUo2fw.png"/>
|
||||
<br/>
|
||||
<strong><a href="https://www.infoq.com/presentations/Twitter-Timeline-Scalability">Source: Twitter timelines at scale</a></strong>
|
||||
</p>
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
*[English](README.md) ∙ [日本語](README-ja.md) ∙ [简体中文](README-zh-Hans.md) ∙ [繁體中文](README-zh-TW.md) | [Brazilian Portuguese](https://github.com/donnemartin/system-design-primer/issues/40) ∙ [Italian](https://github.com/donnemartin/system-design-primer/issues/104) ∙ [Korean](https://github.com/donnemartin/system-design-primer/issues/102) ∙ [Persian](https://github.com/donnemartin/system-design-primer/issues/110) ∙ [Polish](https://github.com/donnemartin/system-design-primer/issues/68) ∙ [Russian](https://github.com/donnemartin/system-design-primer/issues/87) ∙ [Turkish](https://github.com/donnemartin/system-design-primer/issues/39) ∙ [Vietnamese](https://github.com/donnemartin/system-design-primer/issues/127) | [Add Translation](https://github.com/donnemartin/system-design-primer/issues/28)*
|
||||
*[English](README.md) ∙ [日本語](README-ja.md) ∙ [简体中文](README-zh-Hans.md) ∙ [繁體中文](README-zh-TW.md) | [العَرَبِيَّة](https://github.com/donnemartin/system-design-primer/issues/170) ∙ [বাংলা](https://github.com/donnemartin/system-design-primer/issues/220) ∙ [Português do Brasil](https://github.com/donnemartin/system-design-primer/issues/40) ∙ [Deutsch](https://github.com/donnemartin/system-design-primer/issues/186) ∙ [ελληνικά](https://github.com/donnemartin/system-design-primer/issues/130) ∙ [עברית](https://github.com/donnemartin/system-design-primer/issues/272) ∙ [Italiano](https://github.com/donnemartin/system-design-primer/issues/104) ∙ [韓國語](https://github.com/donnemartin/system-design-primer/issues/102) ∙ [فارسی](https://github.com/donnemartin/system-design-primer/issues/110) ∙ [Polski](https://github.com/donnemartin/system-design-primer/issues/68) ∙ [русский язык](https://github.com/donnemartin/system-design-primer/issues/87) ∙ [Español](https://github.com/donnemartin/system-design-primer/issues/136) ∙ [ภาษาไทย](https://github.com/donnemartin/system-design-primer/issues/187) ∙ [Türkçe](https://github.com/donnemartin/system-design-primer/issues/39) ∙ [tiếng Việt](https://github.com/donnemartin/system-design-primer/issues/127) ∙ [Français](https://github.com/donnemartin/system-design-primer/issues/250) | [Add Translation](https://github.com/donnemartin/system-design-primer/issues/28)*
|
||||
|
||||
# 系統設計入門
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/jj3A5N8.png">
|
||||
<img src="http://i.imgur.com/jj3A5N8.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -44,7 +44,7 @@
|
|||
## 學習單字卡
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/zdCAkB3.png">
|
||||
<img src="http://i.imgur.com/zdCAkB3.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -61,7 +61,7 @@
|
|||
你正在尋找資源來面對[**程式語言面試**](https://github.com/donnemartin/interactive-coding-challenges)嗎？
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/b4YtAEN.png">
|
||||
<img src="http://i.imgur.com/b4YtAEN.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -91,7 +91,7 @@
|
|||
> 每一章節都包含更深入資源的連結。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/jrUBAF7.png">
|
||||
<img src="http://i.imgur.com/jrUBAF7.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -435,7 +435,7 @@
|
|||
### CAP 理論
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/bgLMI2u.png">
|
||||
<img src="http://i.imgur.com/bgLMI2u.png"/>
|
||||
<br/>
|
||||
<i><a href=http://robertgreiner.com/2014/08/cap-theorem-revisited>來源：再看 CAP 理論</a></i>
|
||||
</p>
|
||||
|
|
@ -529,7 +529,7 @@ DNS 或是電子郵件系統使用的就是這種方式，最終一致性在高
|
|||
## 域名系統
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/IOyLj4i.jpg">
|
||||
<img src="http://i.imgur.com/IOyLj4i.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/srikrupa5/dns-security-presentation-issa>資料來源：DNS 安全介紹</a></i>
|
||||
</p>
|
||||
|
|
@ -567,7 +567,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
## 內容傳遞網路(CDN)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/h9TAuGI.jpg">
|
||||
<img src="http://i.imgur.com/h9TAuGI.jpg"/>
|
||||
<br/>
|
||||
<i><a href=https://www.creative-artworks.eu/why-use-a-content-delivery-network-cdn/>來源：為什麼要使用 CDN</a></i>
|
||||
</p>
|
||||
|
|
@ -608,7 +608,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
## 負載平衡器
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/h81n9iK.png">
|
||||
<img src="http://i.imgur.com/h81n9iK.png"/>
|
||||
<br/>
|
||||
<i><a href=http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html>來源：可擴展的系統設計模式</a></i>
|
||||
</p>
|
||||
|
|
@ -678,7 +678,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
## 反向代理(網頁伺服器)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/n41Azff.png">
|
||||
<img src="http://i.imgur.com/n41Azff.png"/>
|
||||
<br/>
|
||||
<i><a href=https://upload.wikimedia.org/wikipedia/commons/6/67/Reverse_proxy_h2g2bob.svg>來源：維基百科</a></i>
|
||||
<br/>
|
||||
|
|
@ -721,7 +721,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
## 應用層
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/yB5SYwm.png">
|
||||
<img src="http://i.imgur.com/yB5SYwm.png"/>
|
||||
<br/>
|
||||
<i><a href=http://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer>資料來源：可縮放式系統架構介紹</a></i>
|
||||
</p>
|
||||
|
|
@ -758,7 +758,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
## 資料庫
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/Xkm5CXz.png">
|
||||
<img src="http://i.imgur.com/Xkm5CXz.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.youtube.com/watch?v=vg5onp8TU6Q>來源：擴展你的使用者數量到第一個一千萬量級</a></i>
|
||||
</p>
|
||||
|
|
@ -781,7 +781,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
主資料庫負責讀和寫，並且將寫入的資料複寫至一或多個從屬資料庫中，從屬資料庫只負責讀取。而從屬資料庫可以再將寫入複製到更多以樹狀結構的其他資料庫中。如果主資料庫離線了，系統可以以只讀模式運行，直到某個從屬資料庫被提升為主資料庫，或有新的主資料庫出現。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/C9ioGtn.png">
|
||||
<img src="http://i.imgur.com/C9ioGtn.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>來源： 可擴展性、可用性、穩定性及其模式</a></i>
|
||||
</p>
|
||||
|
|
@ -796,7 +796,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
兩個主要的資料庫都負責讀取和寫入，並且兩者互相協調。如果其中一個主要資料庫離線，系統可以繼續運作。
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/krAHLGg.png">
|
||||
<img src="http://i.imgur.com/krAHLGg.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>來源： 可擴展性、可用性、穩定性及其模式</a></i>
|
||||
</p>
|
||||
|
|
@ -824,7 +824,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
#### 聯邦式資料庫
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/U3qV33e.png">
|
||||
<img src="http://i.imgur.com/U3qV33e.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.youtube.com/watch?v=vg5onp8TU6Q>來源：擴展你的使用者數量到第一個一千萬量級</a></i>
|
||||
</p>
|
||||
|
|
@ -845,7 +845,7 @@ DNS 是階層式的架構，一部分的 DNS 伺服器位於頂層，當查詢
|
|||
#### 分片
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/wU8x5Id.png">
|
||||
<img src="http://i.imgur.com/wU8x5Id.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>來源： 可擴展性、可用性、穩定性及其模式</a></i>
|
||||
</p>
|
||||
|
|
@ -991,7 +991,7 @@ NoSQL 指的是 **鍵-值對的資料庫**、**文件類型資料庫**、**列
|
|||
#### 列儲存型資料庫
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/n16iOGk.png">
|
||||
<img src="http://i.imgur.com/n16iOGk.png"/>
|
||||
<br/>
|
||||
<i><a href=http://blog.grio.com/2015/11/sql-nosql-a-brief-history.html>來源：SQL 和 NoSQL，簡短的歷史介紹</a></i>
|
||||
</p>
|
||||
|
|
@ -1014,7 +1014,7 @@ Google 發表了第一個列儲存型資料庫 [Bigtable](http://www.read.seas.h
|
|||
#### 圖形資料庫
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/fNcl65g.png">
|
||||
<img src="http://i.imgur.com/fNcl65g.png"/>
|
||||
<br/>
|
||||
<i><a href=https://en.wikipedia.org/wiki/File:GraphDatabase_PropertyGraph.png>來源： 圖形化資料庫</a></i>
|
||||
</p>
|
||||
|
|
@ -1042,7 +1042,7 @@ Google 發表了第一個列儲存型資料庫 [Bigtable](http://www.read.seas.h
|
|||
### SQL 或 NoSQL
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/wXGqG5f.png">
|
||||
<img src="http://i.imgur.com/wXGqG5f.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.infoq.com/articles/Transition-RDBMS-NoSQL/>來源：從 RDBMS 轉換到 NoSQL</a></i>
|
||||
</p>
|
||||
|
|
@ -1084,7 +1084,7 @@ Google 發表了第一個列儲存型資料庫 [Bigtable](http://www.read.seas.h
|
|||
## 快取
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/Q6z24La.png">
|
||||
<img src="http://i.imgur.com/Q6z24La.png"/>
|
||||
<br/>
|
||||
<i><a href=http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html>來源：可擴展的系統設計模式</a></i>
|
||||
</p>
|
||||
|
|
@ -1155,7 +1155,7 @@ Redis 還有以下額外的功能：
|
|||
#### 快取模式
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/ONjORqk.png">
|
||||
<img src="http://i.imgur.com/ONjORqk.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast>資料來源：從快取到記憶體資料網格</a></i>
|
||||
</p>
|
||||
|
|
@ -1167,7 +1167,7 @@ Redis 還有以下額外的功能：
|
|||
* 將該筆記錄儲存到快取
|
||||
* 將資料返回
|
||||
|
||||
```
|
||||
```python
|
||||
def get_user(self, user_id):
|
||||
user = cache.get("user.{0}", user_id)
|
||||
if user is None:
|
||||
|
|
@ -1191,7 +1191,7 @@ def get_user(self, user_id):
|
|||
#### 寫入模式
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/0vBc0hN.png">
|
||||
<img src="http://i.imgur.com/0vBc0hN.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>資料來源：可獲展性、可用性、穩定性與模式</a></i>
|
||||
</p>
|
||||
|
|
@ -1210,7 +1210,7 @@ set_user(12345, {"foo":"bar"})
|
|||
|
||||
快取程式碼：
|
||||
|
||||
```
|
||||
```python
|
||||
def set_user(user_id, values):
|
||||
user = db.query("UPDATE Users WHERE id = {0}", user_id, values)
|
||||
cache.set(user_id, user)
|
||||
|
|
@ -1226,7 +1226,7 @@ def set_user(user_id, values):
|
|||
#### 事後寫入(回寫)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/rgSrvjG.png">
|
||||
<img src="http://i.imgur.com/rgSrvjG.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>資料來源：可獲展性、可用性、穩定性與模式</a></i>
|
||||
</p>
|
||||
|
|
@ -1244,7 +1244,7 @@ def set_user(user_id, values):
|
|||
#### 更新式快取
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/kxtjqgE.png">
|
||||
<img src="http://i.imgur.com/kxtjqgE.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast>來源：從快取到記憶體資料網格技術</a></i>
|
||||
</p>
|
||||
|
|
@ -1276,7 +1276,7 @@ def set_user(user_id, values):
|
|||
## 非同步機制
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/54GYsSx.png">
|
||||
<img src="http://i.imgur.com/54GYsSx.png"/>
|
||||
<br/>
|
||||
<i><a href=http://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer>資料來源：可縮放性系統架構介紹</a></i>
|
||||
</p>
|
||||
|
|
@ -1322,7 +1322,7 @@ def set_user(user_id, values):
|
|||
## 通訊
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/5KeocQs.jpg">
|
||||
<img src="http://i.imgur.com/5KeocQs.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.escotal.com/osilayer.html>來源：OSI 七層模型</a></i>
|
||||
</p>
|
||||
|
|
@ -1354,7 +1354,7 @@ HTTP 是依賴於較底層的協議(例如：**TCP** 和 **UDP**) 的應用層
|
|||
### 傳輸控制通訊協定(TCP)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/JdAsdvG.jpg">
|
||||
<img src="http://i.imgur.com/JdAsdvG.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.wildbunny.co.uk/blog/2012/10/09/how-to-make-a-multi-player-game-part-1/>來源：如何開發多人遊戲</a></i>
|
||||
</p>
|
||||
|
|
@ -1378,7 +1378,7 @@ TCP 對於需要高可靠、低時間急迫性的應用來說很有用，比如
|
|||
### 使用者資料流通訊協定 (UDP)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/yzDrJtA.jpg">
|
||||
<img src="http://i.imgur.com/yzDrJtA.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.wildbunny.co.uk/blog/2012/10/09/how-to-make-a-multi-player-game-part-1/>資料來源：如何製作多人遊戲</a></i>
|
||||
</p>
|
||||
|
|
@ -1407,7 +1407,7 @@ UDP 的可靠性較低，但適合用在像是網路電話、視訊聊天、串
|
|||
### 遠端程式呼叫 (RPC)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/iF4Mkb5.png">
|
||||
<img src="http://i.imgur.com/iF4Mkb5.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.puncsky.com/blog/2016-02-13-crack-the-system-design-interview>資料來源：破解系統設計面試</a></i>
|
||||
</p>
|
||||
|
|
@ -1630,7 +1630,7 @@ Notes
|
|||
> 底下是關於真實世界的系統架構是如何設計的文章
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/TcUo2fw.png">
|
||||
<img src="http://i.imgur.com/TcUo2fw.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.infoq.com/presentations/Twitter-Timeline-Scalability>資料來源：可擴展式的 Twitter 時間軸設計</a></i>
|
||||
</p>
|
||||
|
|
|
|||
115
README.md
115
README.md
|
|
@ -1,4 +1,4 @@
|
|||
*[English](README.md) ∙ [日本語](README-ja.md) ∙ [简体中文](README-zh-Hans.md) ∙ [繁體中文](README-zh-TW.md) | [Arabic](https://github.com/donnemartin/system-design-primer/issues/170) ∙ [Bengali](https://github.com/donnemartin/system-design-primer/issues/220) ∙ [Brazilian Portuguese](https://github.com/donnemartin/system-design-primer/issues/40) ∙ [German](https://github.com/donnemartin/system-design-primer/issues/186) ∙ [Greek](https://github.com/donnemartin/system-design-primer/issues/130) ∙ [Italian](https://github.com/donnemartin/system-design-primer/issues/104) ∙ [Korean](https://github.com/donnemartin/system-design-primer/issues/102) ∙ [Persian](https://github.com/donnemartin/system-design-primer/issues/110) ∙ [Polish](https://github.com/donnemartin/system-design-primer/issues/68) ∙ [Russian](https://github.com/donnemartin/system-design-primer/issues/87) ∙ [Spanish](https://github.com/donnemartin/system-design-primer/issues/136) ∙ [Thai](https://github.com/donnemartin/system-design-primer/issues/187) ∙ [Turkish](https://github.com/donnemartin/system-design-primer/issues/39) ∙ [Vietnamese](https://github.com/donnemartin/system-design-primer/issues/127) | [Add Translation](https://github.com/donnemartin/system-design-primer/issues/28)*
|
||||
*[English](README.md) ∙ [日本語](README-ja.md) ∙ [简体中文](README-zh-Hans.md) ∙ [繁體中文](README-zh-TW.md) | [العَرَبِيَّة](https://github.com/donnemartin/system-design-primer/issues/170) ∙ [বাংলা](https://github.com/donnemartin/system-design-primer/issues/220) ∙ [Português do Brasil](https://github.com/donnemartin/system-design-primer/issues/40) ∙ [Deutsch](https://github.com/donnemartin/system-design-primer/issues/186) ∙ [ελληνικά](https://github.com/donnemartin/system-design-primer/issues/130) ∙ [עברית](https://github.com/donnemartin/system-design-primer/issues/272) ∙ [Italiano](https://github.com/donnemartin/system-design-primer/issues/104) ∙ [韓國語](https://github.com/donnemartin/system-design-primer/issues/102) ∙ [فارسی](https://github.com/donnemartin/system-design-primer/issues/110) ∙ [Polski](https://github.com/donnemartin/system-design-primer/issues/68) ∙ [русский язык](https://github.com/donnemartin/system-design-primer/issues/87) ∙ [Español](https://github.com/donnemartin/system-design-primer/issues/136) ∙ [ภาษาไทย](https://github.com/donnemartin/system-design-primer/issues/187) ∙ [Türkçe](https://github.com/donnemartin/system-design-primer/issues/39) ∙ [tiếng Việt](https://github.com/donnemartin/system-design-primer/issues/127) ∙ [Français](https://github.com/donnemartin/system-design-primer/issues/250) | [Add Translation](https://github.com/donnemartin/system-design-primer/issues/28)*
|
||||
|
||||
# The System Design Primer
|
||||
|
||||
|
|
@ -39,7 +39,7 @@ Additional topics for interview prep:
|
|||
## Anki flashcards
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/zdCAkB3.png">
|
||||
<img src="http://i.imgur.com/zdCAkB3.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -56,7 +56,7 @@ Great for use while on-the-go.
|
|||
Looking for resources to help you prep for the [**Coding Interview**](https://github.com/donnemartin/interactive-coding-challenges)?
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/b4YtAEN.png">
|
||||
<img src="http://i.imgur.com/b4YtAEN.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -86,7 +86,7 @@ Review the [Contributing Guidelines](CONTRIBUTING.md).
|
|||
> Each section contains links to more in-depth resources.
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/jrUBAF7.png">
|
||||
<img src="http://i.imgur.com/jrUBAF7.png"/>
|
||||
<br/>
|
||||
</p>
|
||||
|
||||
|
|
@ -107,6 +107,7 @@ Review the [Contributing Guidelines](CONTRIBUTING.md).
|
|||
* [Availability patterns](#availability-patterns)
|
||||
* [Fail-over](#fail-over)
|
||||
* [Replication](#replication)
|
||||
* [Availability in numbers](#availability-in-numbers)
|
||||
* [Domain name system](#domain-name-system)
|
||||
* [Content delivery network](#content-delivery-network)
|
||||
* [Push CDNs](#push-cdns)
|
||||
|
|
@ -431,7 +432,7 @@ Generally, you should aim for **maximal throughput** with **acceptable latency**
|
|||
### CAP theorem
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/bgLMI2u.png">
|
||||
<img src="http://i.imgur.com/bgLMI2u.png"/>
|
||||
<br/>
|
||||
<i><a href=http://robertgreiner.com/2014/08/cap-theorem-revisited>Source: CAP theorem revisited</a></i>
|
||||
</p>
|
||||
|
|
@ -522,10 +523,56 @@ This topic is further discussed in the [Database](#database) section:
|
|||
* [Master-slave replication](#master-slave-replication)
|
||||
* [Master-master replication](#master-master-replication)
|
||||
|
||||
### Availability in numbers
|
||||
|
||||
Availability is often quantified by uptime (or downtime) as a percentage of time the service is available. Availability is generally measured in number of 9s--a service with 99.99% availability is described as having four 9s.
|
||||
|
||||
#### 99.9% availability - three 9s
|
||||
|
||||
| Duration | Acceptable downtime|
|
||||
|---------------------|--------------------|
|
||||
| Downtime per year | 8h 45min 57s |
|
||||
| Downtime per month | 43m 49.7s |
|
||||
| Downtime per week | 10m 4.8s |
|
||||
| Downtime per day | 1m 26.4s |
|
||||
|
||||
#### 99.99% availability - four 9s
|
||||
|
||||
| Duration | Acceptable downtime|
|
||||
|---------------------|--------------------|
|
||||
| Downtime per year | 52min 35.7s |
|
||||
| Downtime per month | 4m 23s |
|
||||
| Downtime per week | 1m 5s |
|
||||
| Downtime per day | 8.6s |
|
||||
|
||||
#### Availability in parallel vs in sequence
|
||||
|
||||
If a service consists of multiple components prone to failure, the service's overall availability depends on whether the components are in sequence or in parallel.
|
||||
|
||||
###### In sequence
|
||||
|
||||
Overall availability decreases when two components with availability < 100% are in sequence:
|
||||
|
||||
```
|
||||
Availability (Total) = Availability (Foo) * Availability (Bar)
|
||||
```
|
||||
|
||||
If both `Foo` and `Bar` each had 99.9% availability, their total availability in sequence would be 99.8%.
|
||||
|
||||
###### In parallel
|
||||
|
||||
Overall availability increases when two components with availability < 100% are in parallel:
|
||||
|
||||
```
|
||||
Availability (Total) = 1 - (1 - Availability (Foo)) * (1 - Availability (Bar))
|
||||
```
|
||||
|
||||
If both `Foo` and `Bar` each had 99.9% availability, their total availability in parallel would be 99.9999%.
|
||||
|
||||
## Domain name system
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/IOyLj4i.jpg">
|
||||
<img src="http://i.imgur.com/IOyLj4i.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/srikrupa5/dns-security-presentation-issa>Source: DNS security presentation</a></i>
|
||||
</p>
|
||||
|
|
@ -563,7 +610,7 @@ Services such as [CloudFlare](https://www.cloudflare.com/dns/) and [Route 53](ht
|
|||
## Content delivery network
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/h9TAuGI.jpg">
|
||||
<img src="http://i.imgur.com/h9TAuGI.jpg"/>
|
||||
<br/>
|
||||
<i><a href=https://www.creative-artworks.eu/why-use-a-content-delivery-network-cdn/>Source: Why use a CDN</a></i>
|
||||
</p>
|
||||
|
|
@ -604,7 +651,7 @@ Sites with heavy traffic work well with pull CDNs, as traffic is spread out more
|
|||
## Load balancer
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/h81n9iK.png">
|
||||
<img src="http://i.imgur.com/h81n9iK.png"/>
|
||||
<br/>
|
||||
<i><a href=http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html>Source: Scalable system design patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -674,7 +721,7 @@ Load balancers can also help with horizontal scaling, improving performance and
|
|||
## Reverse proxy (web server)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/n41Azff.png">
|
||||
<img src="http://i.imgur.com/n41Azff.png"/>
|
||||
<br/>
|
||||
<i><a href=https://upload.wikimedia.org/wikipedia/commons/6/67/Reverse_proxy_h2g2bob.svg>Source: Wikipedia</a></i>
|
||||
<br/>
|
||||
|
|
@ -717,7 +764,7 @@ Additional benefits include:
|
|||
## Application layer
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/yB5SYwm.png">
|
||||
<img src="http://i.imgur.com/yB5SYwm.png"/>
|
||||
<br/>
|
||||
<i><a href=http://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer>Source: Intro to architecting systems for scale</a></i>
|
||||
</p>
|
||||
|
|
@ -752,7 +799,7 @@ Systems such as [Consul](https://www.consul.io/docs/index.html), [Etcd](https://
|
|||
## Database
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/Xkm5CXz.png">
|
||||
<img src="http://i.imgur.com/Xkm5CXz.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.youtube.com/watch?v=w95murBkYmU>Source: Scaling up to your first 10 million users</a></i>
|
||||
</p>
|
||||
|
|
@ -775,7 +822,7 @@ There are many techniques to scale a relational database: **master-slave replica
|
|||
The master serves reads and writes, replicating writes to one or more slaves, which serve only reads. Slaves can also replicate to additional slaves in a tree-like fashion. If the master goes offline, the system can continue to operate in read-only mode until a slave is promoted to a master or a new master is provisioned.
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/C9ioGtn.png">
|
||||
<img src="http://i.imgur.com/C9ioGtn.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -790,7 +837,7 @@ The master serves reads and writes, replicating writes to one or more slaves, wh
|
|||
Both masters serve reads and writes and coordinate with each other on writes. If either master goes down, the system can continue to operate with both reads and writes.
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/krAHLGg.png">
|
||||
<img src="http://i.imgur.com/krAHLGg.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -818,7 +865,7 @@ Both masters serve reads and writes and coordinate with each other on writes. I
|
|||
#### Federation
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/U3qV33e.png">
|
||||
<img src="http://i.imgur.com/U3qV33e.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.youtube.com/watch?v=w95murBkYmU>Source: Scaling up to your first 10 million users</a></i>
|
||||
</p>
|
||||
|
|
@ -839,7 +886,7 @@ Federation (or functional partitioning) splits up databases by function. For ex
|
|||
#### Sharding
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/wU8x5Id.png">
|
||||
<img src="http://i.imgur.com/wU8x5Id.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -983,7 +1030,7 @@ Document stores provide high flexibility and are often used for working with occ
|
|||
#### Wide column store
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/n16iOGk.png">
|
||||
<img src="http://i.imgur.com/n16iOGk.png"/>
|
||||
<br/>
|
||||
<i><a href=http://blog.grio.com/2015/11/sql-nosql-a-brief-history.html>Source: SQL & NoSQL, a brief history</a></i>
|
||||
</p>
|
||||
|
|
@ -1006,7 +1053,7 @@ Wide column stores offer high availability and high scalability. They are often
|
|||
#### Graph database
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/fNcl65g.png">
|
||||
<img src="http://i.imgur.com/fNcl65g.png"/>
|
||||
<br/>
|
||||
<i><a href=https://en.wikipedia.org/wiki/File:GraphDatabase_PropertyGraph.png>Source: Graph database</a></i>
|
||||
</p>
|
||||
|
|
@ -1034,7 +1081,7 @@ Graphs databases offer high performance for data models with complex relationshi
|
|||
### SQL or NoSQL
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/wXGqG5f.png">
|
||||
<img src="http://i.imgur.com/wXGqG5f.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.infoq.com/articles/Transition-RDBMS-NoSQL/>Source: Transitioning from RDBMS to NoSQL</a></i>
|
||||
</p>
|
||||
|
|
@ -1076,7 +1123,7 @@ Sample data well-suited for NoSQL:
|
|||
## Cache
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/Q6z24La.png">
|
||||
<img src="http://i.imgur.com/Q6z24La.png"/>
|
||||
<br/>
|
||||
<i><a href=http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html>Source: Scalable system design patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -1147,7 +1194,7 @@ Since you can only store a limited amount of data in cache, you'll need to deter
|
|||
#### Cache-aside
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/ONjORqk.png">
|
||||
<img src="http://i.imgur.com/ONjORqk.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast>Source: From cache to in-memory data grid</a></i>
|
||||
</p>
|
||||
|
|
@ -1159,7 +1206,7 @@ The application is responsible for reading and writing from storage. The cache
|
|||
* Add entry to cache
|
||||
* Return entry
|
||||
|
||||
```
|
||||
```python
|
||||
def get_user(self, user_id):
|
||||
user = cache.get("user.{0}", user_id)
|
||||
if user is None:
|
||||
|
|
@ -1183,7 +1230,7 @@ Subsequent reads of data added to cache are fast. Cache-aside is also referred
|
|||
#### Write-through
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/0vBc0hN.png">
|
||||
<img src="http://i.imgur.com/0vBc0hN.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -1196,13 +1243,13 @@ The application uses the cache as the main data store, reading and writing data
|
|||
|
||||
Application code:
|
||||
|
||||
```
|
||||
```python
|
||||
set_user(12345, {"foo":"bar"})
|
||||
```
|
||||
|
||||
Cache code:
|
||||
|
||||
```
|
||||
```python
|
||||
def set_user(user_id, values):
|
||||
user = db.query("UPDATE Users WHERE id = {0}", user_id, values)
|
||||
cache.set(user_id, user)
|
||||
|
|
@ -1218,7 +1265,7 @@ Write-through is a slow overall operation due to the write operation, but subseq
|
|||
#### Write-behind (write-back)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/rgSrvjG.png">
|
||||
<img src="http://i.imgur.com/rgSrvjG.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/jboner/scalability-availability-stability-patterns/>Source: Scalability, availability, stability, patterns</a></i>
|
||||
</p>
|
||||
|
|
@ -1236,7 +1283,7 @@ In write-behind, the application does the following:
|
|||
#### Refresh-ahead
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/kxtjqgE.png">
|
||||
<img src="http://i.imgur.com/kxtjqgE.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast>Source: From cache to in-memory data grid</a></i>
|
||||
</p>
|
||||
|
|
@ -1268,7 +1315,7 @@ Refresh-ahead can result in reduced latency vs read-through if the cache can acc
|
|||
## Asynchronism
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/54GYsSx.png">
|
||||
<img src="http://i.imgur.com/54GYsSx.png"/>
|
||||
<br/>
|
||||
<i><a href=http://lethain.com/introduction-to-architecting-systems-for-scale/#platform_layer>Source: Intro to architecting systems for scale</a></i>
|
||||
</p>
|
||||
|
|
@ -1314,7 +1361,7 @@ If queues start to grow significantly, the queue size can become larger than mem
|
|||
## Communication
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/5KeocQs.jpg">
|
||||
<img src="http://i.imgur.com/5KeocQs.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.escotal.com/osilayer.html>Source: OSI 7 layer model</a></i>
|
||||
</p>
|
||||
|
|
@ -1346,7 +1393,7 @@ HTTP is an application layer protocol relying on lower-level protocols such as *
|
|||
### Transmission control protocol (TCP)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/JdAsdvG.jpg">
|
||||
<img src="http://i.imgur.com/JdAsdvG.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.wildbunny.co.uk/blog/2012/10/09/how-to-make-a-multi-player-game-part-1/>Source: How to make a multiplayer game</a></i>
|
||||
</p>
|
||||
|
|
@ -1370,7 +1417,7 @@ Use TCP over UDP when:
|
|||
### User datagram protocol (UDP)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/yzDrJtA.jpg">
|
||||
<img src="http://i.imgur.com/yzDrJtA.jpg"/>
|
||||
<br/>
|
||||
<i><a href=http://www.wildbunny.co.uk/blog/2012/10/09/how-to-make-a-multi-player-game-part-1/>Source: How to make a multiplayer game</a></i>
|
||||
</p>
|
||||
|
|
@ -1399,7 +1446,7 @@ Use UDP over TCP when:
|
|||
### Remote procedure call (RPC)
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/iF4Mkb5.png">
|
||||
<img src="http://i.imgur.com/iF4Mkb5.png"/>
|
||||
<br/>
|
||||
<i><a href=http://www.puncsky.com/blog/2016-02-13-crack-the-system-design-interview>Source: Crack the system design interview</a></i>
|
||||
</p>
|
||||
|
|
@ -1595,7 +1642,7 @@ Handy metrics based on numbers above:
|
|||
| Question | Reference(s) |
|
||||
|---|---|
|
||||
| Design a file sync service like Dropbox | [youtube.com](https://www.youtube.com/watch?v=PE4gwstWhmc) |
|
||||
| Design a search engine like Google | [queue.acm.org](http://queue.acm.org/detail.cfm?id=988407)<br/>[stackexchange.com](http://programmers.stackexchange.com/questions/38324/interview-question-how-would-you-implement-google-search)<br/>[ardendertat.com](http://www.ardendertat.com/2012/01/11/implementing-search-engines/)<br>[stanford.edu](http://infolab.stanford.edu/~backrub/google.html) |
|
||||
| Design a search engine like Google | [queue.acm.org](http://queue.acm.org/detail.cfm?id=988407)<br/>[stackexchange.com](http://programmers.stackexchange.com/questions/38324/interview-question-how-would-you-implement-google-search)<br/>[ardendertat.com](http://www.ardendertat.com/2012/01/11/implementing-search-engines/)<br/>[stanford.edu](http://infolab.stanford.edu/~backrub/google.html) |
|
||||
| Design a scalable web crawler like Google | [quora.com](https://www.quora.com/How-can-I-build-a-web-crawler-from-scratch) |
|
||||
| Design Google docs | [code.google.com](https://code.google.com/p/google-mobwrite/)<br/>[neil.fraser.name](https://neil.fraser.name/writing/sync/) |
|
||||
| Design a key-value store like Redis | [slideshare.net](http://www.slideshare.net/dvirsky/introduction-to-redis) |
|
||||
|
|
@ -1623,7 +1670,7 @@ Handy metrics based on numbers above:
|
|||
> Articles on how real world systems are designed.
|
||||
|
||||
<p align="center">
|
||||
<img src="http://i.imgur.com/TcUo2fw.png">
|
||||
<img src="http://i.imgur.com/TcUo2fw.png"/>
|
||||
<br/>
|
||||
<i><a href=https://www.infoq.com/presentations/Twitter-Timeline-Scalability>Source: Twitter timelines at scale</a></i>
|
||||
</p>
|
||||
|
|
@ -1650,7 +1697,7 @@ Handy metrics based on numbers above:
|
|||
| Data store | **Redis** - Distributed memory caching system with persistence and value types | [slideshare.net](http://www.slideshare.net/dvirsky/introduction-to-redis) |
|
||||
| | | |
|
||||
| File system | **Google File System (GFS)** - Distributed file system | [research.google.com](http://static.googleusercontent.com/media/research.google.com/zh-CN/us/archive/gfs-sosp2003.pdf) |
|
||||
| File system | **Hadoop File System (HDFS)** - Open source implementation of GFS | [apache.org](https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html) |
|
||||
| File system | **Hadoop File System (HDFS)** - Open source implementation of GFS | [apache.org](http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html) |
|
||||
| | | |
|
||||
| Misc | **Chubby** - Lock service for loosely-coupled distributed systems from Google | [research.google.com](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/chubby-osdi06.pdf) |
|
||||
| Misc | **Dapper** - Distributed systems tracing infrastructure | [research.google.com](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36356.pdf)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
title: System Design Primer
|
||||
creator: Donne Martin
|
||||
date: 2018
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
#! /usr/bin/env sh
|
||||
|
||||
generate_from_stdin() {
|
||||
outfile=$1
|
||||
language=$2
|
||||
|
||||
echo "Generating '$language' ..."
|
||||
|
||||
pandoc --metadata-file=epub-metadata.yaml --metadata=lang:$2 --from=markdown -o $1 <&0
|
||||
|
||||
echo "Done! You can find the '$language' book at ./$outfile"
|
||||
}
|
||||
|
||||
generate_with_solutions () {
|
||||
tmpfile=$(mktemp /tmp/sytem-design-primer-epub-generator.XXX)
|
||||
|
||||
cat ./README.md >> $tmpfile
|
||||
|
||||
for dir in ./solutions/system_design/*; do
|
||||
case $dir in *template*) continue;; esac
|
||||
case $dir in *__init__.py*) continue;; esac
|
||||
: [[ -d "$dir" ]] && ( cd "$dir" && cat ./README.md >> $tmpfile && echo "" >> $tmpfile )
|
||||
done
|
||||
|
||||
cat $tmpfile | generate_from_stdin 'README.epub' 'en'
|
||||
|
||||
rm "$tmpfile"
|
||||
}
|
||||
|
||||
generate () {
|
||||
name=$1
|
||||
language=$2
|
||||
|
||||
cat $name.md | generate_from_stdin $name.epub $language
|
||||
}
|
||||
|
||||
generate_with_solutions
|
||||
generate README-ja ja
|
||||
generate README-zh-Hans zh-Hans
|
||||
generate README-zh-TW zh-TW
|
||||
|
|
@ -182,7 +182,7 @@ For the **Category Service**, we can seed a seller-to-category dictionary with t
|
|||
|
||||
**Clarify with your interviewer how much code you are expected to write**.
|
||||
|
||||
```
|
||||
```python
|
||||
class DefaultCategories(Enum):
|
||||
|
||||
HOUSING = 0
|
||||
|
|
@ -199,7 +199,7 @@ seller_category_map['Target'] = DefaultCategories.SHOPPING
|
|||
|
||||
For sellers not initially seeded in the map, we could use a crowdsourcing effort by evaluating the manual category overrides our users provide. We could use a heap to quickly lookup the top manual override per seller in O(1) time.
|
||||
|
||||
```
|
||||
```python
|
||||
class Categorizer(object):
|
||||
|
||||
def __init__(self, seller_category_map, self.seller_category_crowd_overrides_map):
|
||||
|
|
@ -219,7 +219,7 @@ class Categorizer(object):
|
|||
|
||||
Transaction implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class Transaction(object):
|
||||
|
||||
def __init__(self, created_at, seller, amount):
|
||||
|
|
@ -232,7 +232,7 @@ class Transaction(object):
|
|||
|
||||
To start, we could use a generic budget template that allocates category amounts based on income tiers. Using this approach, we would not have to store the 100 million budget items identified in the constraints, only those that the user overrides. If a user overrides a budget category, which we could store the override in the `TABLE budget_overrides`.
|
||||
|
||||
```
|
||||
```python
|
||||
class Budget(object):
|
||||
|
||||
def __init__(self, income):
|
||||
|
|
@ -273,7 +273,7 @@ user_id timestamp seller amount
|
|||
|
||||
**MapReduce** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class SpendingByCategory(MRJob):
|
||||
|
||||
def __init__(self, categorizer):
|
||||
|
|
|
|||
|
|
@ -0,0 +1,330 @@
|
|||
# 设计 Pastebin.com (或者 Bit.ly)
|
||||
|
||||
**Note: 为了避免重复，当前文档直接链接到[系统设计主题](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#系统设计主题的索引)的相关区域，请参考链接内容以获得综合的讨论点、权衡和替代方案。**
|
||||
|
||||
**设计 Bit.ly** - 是一个类似的问题，区别是 pastebin 需要存储的是 paste 的内容，而不是原始的未短化的 url。
|
||||
|
||||
## 第一步：概述用例和约束
|
||||
|
||||
> 收集这个问题的需求和范畴。
|
||||
> 问相关问题来明确用例和约束。
|
||||
> 讨论一些假设。
|
||||
|
||||
因为没有面试官来明确这些问题，所以我们自己将定义一些用例和约束。
|
||||
|
||||
### 用例
|
||||
|
||||
#### 我们将问题的范畴限定在如下用例
|
||||
|
||||
* **用户** 输入一段文本，然后得到一个随机生成的链接
|
||||
* 过期设置
|
||||
* 默认的设置是不会过期的
|
||||
* 可以选择设置一个过期的时间
|
||||
* **用户** 输入一个 paste 的 url 后，可以看到它存储的内容
|
||||
* **用户** 是匿名的
|
||||
* **Service** 跟踪页面分析
|
||||
* 一个月的访问统计
|
||||
* **Service** 删除过期的 pastes
|
||||
* **Service** 需要高可用
|
||||
|
||||
#### 超出范畴的用例
|
||||
|
||||
* **用户** 可以注册一个账户
|
||||
* **用户** 通过验证邮箱
|
||||
* **用户** 可以用注册的账户登录
|
||||
* **用户** 可以编辑文档
|
||||
* **用户** 可以设置可见性
|
||||
* **用户** 可以设置短链接
|
||||
|
||||
### 约束和假设
|
||||
|
||||
#### 状态假设
|
||||
|
||||
* 访问流量不是均匀分布的
|
||||
* 打开一个短链接应该是很快的
|
||||
* pastes 只能是文本
|
||||
* 页面访问分析数据可以不用实时
|
||||
* 一千万的用户量
|
||||
* 每个月一千万的 paste 写入量
|
||||
* 每个月一亿的 paste 读取量
|
||||
* 读写比例在 10:1
|
||||
|
||||
#### 计算使用
|
||||
|
||||
**向面试官说明你是否应该粗略计算一下使用情况。**
|
||||
|
||||
* 每个 paste 的大小
|
||||
* 每一个 paste 1 KB
|
||||
* `shortlink` - 7 bytes
|
||||
* `expiration_length_in_minutes` - 4 bytes
|
||||
* `created_at` - 5 bytes
|
||||
* `paste_path` - 255 bytes
|
||||
* 总共 = ~1.27 KB
|
||||
* 每个月新的 paste 内容在 12.7GB
|
||||
* (1.27 * 10000000)KB / 月的 paste
|
||||
* 三年内将近 450GB 的新 paste 内容
|
||||
* 三年内 3.6 亿短链接
|
||||
* 假设大部分都是新的 paste，而不是需要更新已存在的 paste
|
||||
* 平均 4paste/s 的写入速度
|
||||
* 平均 40paste/s 的读取速度
|
||||
|
||||
简单的转换指南:
|
||||
|
||||
* 2.5 百万 req/s
|
||||
* 1 req/s = 2.5 百万 req/m
|
||||
* 40 req/s = 1 亿 req/m
|
||||
* 400 req/s = 10 亿 req/m
|
||||
|
||||
## 第二步：创建一个高层次设计
|
||||
|
||||
> 概述一个包括所有重要的组件的高层次设计
|
||||
|
||||
|
||||
|
||||
## 第三步：设计核心组件
|
||||
|
||||
> 深入每一个核心组件的细节
|
||||
|
||||
### 用例：用户输入一段文本，然后得到一个随机生成的链接
|
||||
|
||||
我们可以用一个 [关系型数据库](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#关系型数据库管理系统rdbms)作为一个大的哈希表，用来把生成的 url 映射到一个包含 paste 文件的文件服务器和路径上。
|
||||
|
||||
为了避免托管一个文件服务器，我们可以用一个托管的**对象存储**，比如 Amazon 的 S3 或者[NoSQL 文档类型存储](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#文档类型存储)。
|
||||
|
||||
作为一个大的哈希表的关系型数据库的替代方案，我们可以用[NoSQL 键值存储](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#键-值存储)。我们需要讨论[选择 SQL 或 NoSQL 之间的权衡](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#sql-还是-nosql)。下面的讨论是使用关系型数据库方法。
|
||||
|
||||
* **客户端** 发送一个创建 paste 的请求到作为一个[反向代理](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#反向代理web-服务器)启动的 **Web 服务器**。
|
||||
* **Web 服务器** 转发请求给 **写接口** 服务器
|
||||
* **写接口** 服务器执行如下操作：
|
||||
* 生成一个唯一的 url
|
||||
* 检查这个 url 在 **SQL 数据库** 里面是否是唯一的
|
||||
* 如果这个 url 不是唯一的，生成另外一个 url
|
||||
* 如果我们支持自定义 url，我们可以使用用户提供的 url（也需要检查是否重复）
|
||||
* 把生成的 url 存储到 **SQL 数据库** 的 `pastes` 表里面
|
||||
* 存储 paste 的内容数据到 **对象存储** 里面
|
||||
* 返回生成的 url
|
||||
|
||||
**向面试官阐明你需要写多少代码**
|
||||
|
||||
`pastes` 表可以有如下结构：
|
||||
|
||||
```sql
|
||||
shortlink char(7) NOT NULL
|
||||
expiration_length_in_minutes int NOT NULL
|
||||
created_at datetime NOT NULL
|
||||
paste_path varchar(255) NOT NULL
|
||||
PRIMARY KEY(shortlink)
|
||||
```
|
||||
|
||||
我们将在 `shortlink` 字段和 `created_at` 字段上创建一个[数据库索引](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#使用正确的索引)，用来提高查询的速度（避免因为扫描全表导致的长时间查询）并将数据保存在内存中，从内存里面顺序读取 1MB 的数据需要大概 250 微秒，而从 SSD 上读取则需要花费 4 倍的时间，从硬盘上则需要花费 80 倍的时间。<sup><a href=https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#每个程序员都应该知道的延迟数 > 1</a></sup>
|
||||
|
||||
为了生成唯一的 url，我们可以：
|
||||
|
||||
* 使用 [**MD5**](https://en.wikipedia.org/wiki/MD5) 来哈希用户的 IP 地址 + 时间戳
|
||||
* MD5 是一个普遍用来生成一个 128-bit 长度的哈希值的一种哈希方法
|
||||
* MD5 是一致分布的
|
||||
* 或者我们也可以用 MD5 哈希一个随机生成的数据
|
||||
* 用 [**Base 62**](https://www.kerstner.at/2012/07/shortening-strings-using-base-62-encoding/) 编码 MD5 哈希值
|
||||
* 对于 urls，使用 Base 62 编码 `[a-zA-Z0-9]` 是比较合适的
|
||||
* 对于每一个原始输入只会有一个 hash 结果，Base 62 是确定的（不涉及随机性）
|
||||
* Base 64 是另外一个流行的编码方案，但是对于 urls，会因为额外的 `+` 和 `-` 字符串而产生一些问题
|
||||
* 以下 [Base 62 伪代码](http://stackoverflow.com/questions/742013/how-to-code-a-url-shortener) 执行的时间复杂度是 O(k)，k 是数字的数量 = 7：
|
||||
|
||||
```python
|
||||
def base_encode(num, base=62):
|
||||
digits = []
|
||||
while num > 0
|
||||
remainder = modulo(num, base)
|
||||
digits.push(remainder)
|
||||
num = divide(num, base)
|
||||
digits = digits.reverse
|
||||
```
|
||||
|
||||
* 取输出的前 7 个字符，结果会有 62^7 个可能的值，应该足以满足在 3 年内处理 3.6 亿个短链接的约束：
|
||||
|
||||
```python
|
||||
url = base_encode(md5(ip_address+timestamp))[:URL_LENGTH]
|
||||
```
|
||||
|
||||
我们将会用一个公开的 [**REST 风格接口**](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#表述性状态转移rest)：
|
||||
|
||||
```shell
|
||||
$ curl -X POST --data '{"expiration_length_in_minutes":"60", \"paste_contents":"Hello World!"}' https://pastebin.com/api/v1/paste
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"shortlink": "foobar"
|
||||
}
|
||||
```
|
||||
|
||||
用于内部通信，我们可以用 [RPC](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#远程过程调用协议rpc)。
|
||||
|
||||
### 用例：用户输入一个 paste 的 url 后可以看到它存储的内容
|
||||
|
||||
* **客户端** 发送一个获取 paste 请求到 **Web Server**
|
||||
* **Web Server** 转发请求给 **读取接口** 服务器
|
||||
* **读取接口** 服务器执行如下操作：
|
||||
* 在 **SQL 数据库** 检查这个生成的 url
|
||||
* 如果这个 url 在 **SQL 数据库** 里面，则从 **对象存储** 获取这个 paste 的内容
|
||||
* 否则，返回一个错误页面给用户
|
||||
|
||||
REST API：
|
||||
|
||||
```shell
|
||||
curl https://pastebin.com/api/v1/paste?shortlink=foobar
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"paste_contents": "Hello World",
|
||||
"created_at": "YYYY-MM-DD HH:MM:SS",
|
||||
"expiration_length_in_minutes": "60"
|
||||
}
|
||||
```
|
||||
|
||||
### 用例： 服务跟踪分析页面
|
||||
|
||||
因为实时分析不是必须的，所以我们可以简单的 **MapReduce** **Web Server** 的日志，用来生成点击次数。
|
||||
|
||||
```python
|
||||
class HitCounts(MRJob):
|
||||
|
||||
def extract_url(self, line):
|
||||
"""Extract the generated url from the log line."""
|
||||
...
|
||||
|
||||
def extract_year_month(self, line):
|
||||
"""Return the year and month portions of the timestamp."""
|
||||
...
|
||||
|
||||
def mapper(self, _, line):
|
||||
"""Parse each log line, extract and transform relevant lines.
|
||||
|
||||
Emit key value pairs of the form:
|
||||
|
||||
(2016-01, url0), 1
|
||||
(2016-01, url0), 1
|
||||
(2016-01, url1), 1
|
||||
"""
|
||||
url = self.extract_url(line)
|
||||
period = self.extract_year_month(line)
|
||||
yield (period, url), 1
|
||||
|
||||
def reducer(self, key, values):
|
||||
"""Sum values for each key.
|
||||
|
||||
(2016-01, url0), 2
|
||||
(2016-01, url1), 1
|
||||
"""
|
||||
yield key, sum(values)
|
||||
```
|
||||
|
||||
### 用例： 服务删除过期的 pastes
|
||||
|
||||
为了删除过期的 pastes，我们可以直接搜索 **SQL 数据库** 中所有的过期时间比当前时间更早的记录，
|
||||
所有过期的记录将从这张表里面删除（或者将其标记为过期）。
|
||||
|
||||
## 第四步：扩展这个设计
|
||||
|
||||
> 给定约束条件，识别和解决瓶颈。
|
||||
|
||||
|
||||
|
||||
**重要提示: 不要简单的从最初的设计直接跳到最终的设计**
|
||||
|
||||
说明您将迭代地执行这样的操作：1)**Benchmark/Load 测试**，2)**Profile** 出瓶颈，3)在评估替代方案和权衡时解决瓶颈，4)重复前面，可以参考[在 AWS 上设计一个可以支持百万用户的系统](../scaling_aws/README.md)这个用来解决如何迭代地扩展初始设计的例子。
|
||||
|
||||
重要的是讨论在初始设计中可能遇到的瓶颈，以及如何解决每个瓶颈。比如，在多个 **Web 服务器** 上添加 **负载平衡器** 可以解决哪些问题？ **CDN** 解决哪些问题？**Master-Slave Replicas** 解决哪些问题? 替代方案是什么和怎么对每一个替代方案进行权衡比较？
|
||||
|
||||
我们将介绍一些组件来完成设计，并解决可伸缩性问题。内部的负载平衡器并不能减少杂乱。
|
||||
|
||||
**为了避免重复的讨论**， 参考以下[系统设计主题](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#系统设计主题的索引)获取主要讨论要点、权衡和替代方案：
|
||||
|
||||
* [DNS](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#域名系统)
|
||||
* [CDN](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#内容分发网络cdn)
|
||||
* [负载均衡器](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#负载均衡器)
|
||||
* [水平扩展](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#水平扩展)
|
||||
* [反向代理（web 服务器）](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#反向代理web-服务器)
|
||||
* [应用层](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#应用层)
|
||||
* [缓存](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#缓存)
|
||||
* [关系型数据库管理系统 (RDBMS)](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#关系型数据库管理系统rdbms)
|
||||
* [SQL write master-slave failover](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#故障切换)
|
||||
* [主从复制](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#主从复制)
|
||||
* [一致性模式](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#一致性模式)
|
||||
* [可用性模式](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#可用性模式)
|
||||
|
||||
**分析存储数据库** 可以用比如 Amazon Redshift 或者 Google BigQuery 这样的数据仓库解决方案。
|
||||
|
||||
一个像 Amazon S3 这样的 **对象存储**，可以轻松处理每月 12.7 GB 的新内容约束。
|
||||
|
||||
要处理 *平均* 每秒 40 读请求(峰值更高)，其中热点内容的流量应该由 **内存缓存** 处理，而不是数据库。**内存缓存** 对于处理分布不均匀的流量和流量峰值也很有用。只要副本没有陷入复制写的泥潭，**SQL Read Replicas** 应该能够处理缓存丢失。
|
||||
|
||||
对于单个 **SQL Write Master-Slave**，*平均* 每秒 4paste 写入 (峰值更高) 应该是可以做到的。否则，我们需要使用额外的 SQL 扩展模式:
|
||||
|
||||
* [联合](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#联合)
|
||||
* [分片](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#分片)
|
||||
* [非规范化](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#非规范化)
|
||||
* [SQL 调优](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#SQL调优)
|
||||
|
||||
我们还应该考虑将一些数据移动到 **NoSQL 数据库**。
|
||||
|
||||
## 额外的话题
|
||||
|
||||
> 是否更深入探讨额外主题，取决于问题的范围和面试剩余的时间。
|
||||
|
||||
### NoSQL
|
||||
|
||||
* [键值存储](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#键-值存储)
|
||||
* [文档存储](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#文档类型存储)
|
||||
* [列型存储](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#列型存储)
|
||||
* [图数据库](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#图数据库)
|
||||
* [sql 还是 nosql](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#sql-还是-nosql)
|
||||
|
||||
### 缓存
|
||||
|
||||
* 在哪缓存
|
||||
* [客户端缓存](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#客户端缓存)
|
||||
* [CDN 缓存](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#cdn-缓存)
|
||||
* [Web 服务器缓存](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#web-服务器缓存)
|
||||
* [数据库缓存](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#数据库缓存)
|
||||
* [应用缓存](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#应用缓存)
|
||||
* 缓存什么
|
||||
* [数据库查询级别的缓存](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#数据库查询级别的缓存)
|
||||
* [对象级别的缓存](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#对象级别的缓存)
|
||||
* 何时更新缓存
|
||||
* [缓存模式](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#缓存模式)
|
||||
* [直写模式](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#直写模式)
|
||||
* [回写模式](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#回写模式)
|
||||
* [刷新](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#刷新)
|
||||
|
||||
### 异步和微服务
|
||||
|
||||
* [消息队列](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#消息队列)
|
||||
* [任务队列](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#任务队列)
|
||||
* [背压](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#背压)
|
||||
* [微服务](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#微服务)
|
||||
|
||||
### 通信
|
||||
|
||||
* 讨论权衡:
|
||||
* 跟客户端之间的外部通信 - [HTTP APIs following REST](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#表述性状态转移rest)
|
||||
* 内部通信 - [RPC](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#远程过程调用协议rpc)
|
||||
* [服务发现](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#服务发现)
|
||||
|
||||
### 安全
|
||||
|
||||
参考[安全](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#安全)。
|
||||
|
||||
### 延迟数字
|
||||
|
||||
见[每个程序员都应该知道的延迟数](https://github.com/donnemartin/system-design-primer/blob/master/README-zh-Hans.md#每个程序员都应该知道的延迟数)。
|
||||
|
||||
### 持续进行
|
||||
|
||||
* 继续对系统进行基准测试和监控，以在瓶颈出现时解决它们
|
||||
* 扩展是一个迭代的过程
|
||||
|
|
@ -130,7 +130,7 @@ To generate the unique url, we could:
|
|||
* Base 64 is another popular encoding but provides issues for urls because of the additional `+` and `/` characters
|
||||
* The following [Base 62 pseudocode](http://stackoverflow.com/questions/742013/how-to-code-a-url-shortener) runs in O(k) time where k is the number of digits = 7:
|
||||
|
||||
```
|
||||
```python
|
||||
def base_encode(num, base=62):
|
||||
digits = []
|
||||
while num > 0
|
||||
|
|
@ -142,7 +142,7 @@ def base_encode(num, base=62):
|
|||
|
||||
* Take the first 7 characters of the output, which results in 62^7 possible values and should be sufficient to handle our constraint of 360 million shortlinks in 3 years:
|
||||
|
||||
```
|
||||
```python
|
||||
url = base_encode(md5(ip_address+timestamp))[:URL_LENGTH]
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -97,7 +97,7 @@ The cache can use a doubly-linked list: new items will be added to the head whil
|
|||
|
||||
**Query API Server** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class QueryApi(object):
|
||||
|
||||
def __init__(self, memory_cache, reverse_index_service):
|
||||
|
|
@ -121,7 +121,7 @@ class QueryApi(object):
|
|||
|
||||
**Node** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class Node(object):
|
||||
|
||||
def __init__(self, query, results):
|
||||
|
|
@ -131,7 +131,7 @@ class Node(object):
|
|||
|
||||
**LinkedList** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class LinkedList(object):
|
||||
|
||||
def __init__(self):
|
||||
|
|
@ -150,7 +150,7 @@ class LinkedList(object):
|
|||
|
||||
**Cache** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class Cache(object):
|
||||
|
||||
def __init__(self, MAX_SIZE):
|
||||
|
|
|
|||
|
|
@ -102,7 +102,7 @@ We'll use a multi-step **MapReduce**:
|
|||
* **Step 1** - Transform the data to `(category, product_id), sum(quantity)`
|
||||
* **Step 2** - Perform a distributed sort
|
||||
|
||||
```
|
||||
```python
|
||||
class SalesRanker(MRJob):
|
||||
|
||||
def within_past_week(self, timestamp):
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ Handy conversion guide:
|
|||
|
||||
Without the constraint of millions of users (vertices) and billions of friend relationships (edges), we could solve this unweighted shortest path task with a general BFS approach:
|
||||
|
||||
```
|
||||
```python
|
||||
class Graph(Graph):
|
||||
|
||||
def shortest_path(self, source, dest):
|
||||
|
|
@ -117,7 +117,7 @@ We won't be able to fit all users on the same machine, we'll need to [shard](htt
|
|||
|
||||
**Lookup Service** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class LookupService(object):
|
||||
|
||||
def __init__(self):
|
||||
|
|
@ -132,7 +132,7 @@ class LookupService(object):
|
|||
|
||||
**Person Server** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class PersonServer(object):
|
||||
|
||||
def __init__(self):
|
||||
|
|
@ -151,7 +151,7 @@ class PersonServer(object):
|
|||
|
||||
**Person** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class Person(object):
|
||||
|
||||
def __init__(self, id, name, friend_ids):
|
||||
|
|
@ -162,7 +162,7 @@ class Person(object):
|
|||
|
||||
**User Graph Service** implementation:
|
||||
|
||||
```
|
||||
```python
|
||||
class UserGraphService(object):
|
||||
|
||||
def __init__(self, lookup_service):
|
||||
|
|
|
|||
|
|
@ -100,7 +100,7 @@ We could store `links_to_crawl` and `crawled_links` in a key-value **NoSQL Datab
|
|||
|
||||
`PagesDataStore` is an abstraction within the **Crawler Service** that uses the **NoSQL Database**:
|
||||
|
||||
```
|
||||
```python
|
||||
class PagesDataStore(object):
|
||||
|
||||
def __init__(self, db);
|
||||
|
|
@ -134,7 +134,7 @@ class PagesDataStore(object):
|
|||
|
||||
`Page` is an abstraction within the **Crawler Service** that encapsulates a page, its contents, child urls, and signature:
|
||||
|
||||
```
|
||||
```python
|
||||
class Page(object):
|
||||
|
||||
def __init__(self, url, contents, child_urls, signature):
|
||||
|
|
@ -146,7 +146,7 @@ class Page(object):
|
|||
|
||||
`Crawler` is the main class within **Crawler Service**, composed of `Page` and `PagesDataStore`.
|
||||
|
||||
```
|
||||
```python
|
||||
class Crawler(object):
|
||||
|
||||
def __init__(self, data_store, reverse_index_queue, doc_index_queue):
|
||||
|
|
@ -187,7 +187,7 @@ We'll want to remove duplicate urls:
|
|||
* For smaller lists we could use something like `sort | unique`
|
||||
* With 1 billion links to crawl, we could use **MapReduce** to output only entries that have a frequency of 1
|
||||
|
||||
```
|
||||
```python
|
||||
class RemoveDuplicateUrls(MRJob):
|
||||
|
||||
def mapper(self, _, line):
|
||||
|
|
|
|||
Loading…
Reference in New Issue