From ebd9a6a771351a2dfe1b443da69e49057f249071 Mon Sep 17 00:00:00 2001 From: Vijeta Hingorani Date: Mon, 5 Sep 2022 10:35:47 -0400 Subject: [PATCH] Adding DDoS preventing techniques --- README.md | 255 ++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 173 insertions(+), 82 deletions(-) diff --git a/README.md b/README.md index 9f0be486..00b8d38e 100644 --- a/README.md +++ b/README.md @@ -97,87 +97,178 @@ Review the [Contributing Guidelines](CONTRIBUTING.md).

-* [System design topics: start here](#system-design-topics-start-here) - * [Step 1: Review the scalability video lecture](#step-1-review-the-scalability-video-lecture) - * [Step 2: Review the scalability article](#step-2-review-the-scalability-article) - * [Next steps](#next-steps) -* [Performance vs scalability](#performance-vs-scalability) -* [Latency vs throughput](#latency-vs-throughput) -* [Availability vs consistency](#availability-vs-consistency) - * [CAP theorem](#cap-theorem) - * [CP - consistency and partition tolerance](#cp---consistency-and-partition-tolerance) - * [AP - availability and partition tolerance](#ap---availability-and-partition-tolerance) -* [Consistency patterns](#consistency-patterns) - * [Weak consistency](#weak-consistency) - * [Eventual consistency](#eventual-consistency) - * [Strong consistency](#strong-consistency) -* [Availability patterns](#availability-patterns) - * [Fail-over](#fail-over) - * [Replication](#replication) - * [Availability in numbers](#availability-in-numbers) -* [Domain name system](#domain-name-system) -* [Content delivery network](#content-delivery-network) - * [Push CDNs](#push-cdns) - * [Pull CDNs](#pull-cdns) -* [Load balancer](#load-balancer) - * [Active-passive](#active-passive) - * [Active-active](#active-active) - * [Layer 4 load balancing](#layer-4-load-balancing) - * [Layer 7 load balancing](#layer-7-load-balancing) - * [Horizontal scaling](#horizontal-scaling) -* [Reverse proxy (web server)](#reverse-proxy-web-server) - * [Load balancer vs reverse proxy](#load-balancer-vs-reverse-proxy) -* [Application layer](#application-layer) - * [Microservices](#microservices) - * [Service discovery](#service-discovery) -* [Database](#database) - * [Relational database management system (RDBMS)](#relational-database-management-system-rdbms) - * [Master-slave replication](#master-slave-replication) - * [Master-master replication](#master-master-replication) - * [Federation](#federation) - * [Sharding](#sharding) - * [Denormalization](#denormalization) - * [SQL tuning](#sql-tuning) - * [NoSQL](#nosql) - * [Key-value store](#key-value-store) - * [Document store](#document-store) - * [Wide column store](#wide-column-store) - * [Graph Database](#graph-database) - * [SQL or NoSQL](#sql-or-nosql) -* [Cache](#cache) - * [Client caching](#client-caching) - * [CDN caching](#cdn-caching) - * [Web server caching](#web-server-caching) - * [Database caching](#database-caching) - * [Application caching](#application-caching) - * [Caching at the database query level](#caching-at-the-database-query-level) - * [Caching at the object level](#caching-at-the-object-level) - * [When to update the cache](#when-to-update-the-cache) - * [Cache-aside](#cache-aside) - * [Write-through](#write-through) - * [Write-behind (write-back)](#write-behind-write-back) - * [Refresh-ahead](#refresh-ahead) -* [Asynchronism](#asynchronism) - * [Message queues](#message-queues) - * [Task queues](#task-queues) - * [Back pressure](#back-pressure) -* [Communication](#communication) - * [Transmission control protocol (TCP)](#transmission-control-protocol-tcp) - * [User datagram protocol (UDP)](#user-datagram-protocol-udp) - * [Remote procedure call (RPC)](#remote-procedure-call-rpc) - * [Representational state transfer (REST)](#representational-state-transfer-rest) -* [Security](#security) -* [Appendix](#appendix) - * [Powers of two table](#powers-of-two-table) - * [Latency numbers every programmer should know](#latency-numbers-every-programmer-should-know) - * [Additional system design interview questions](#additional-system-design-interview-questions) - * [Real world architectures](#real-world-architectures) - * [Company architectures](#company-architectures) - * [Company engineering blogs](#company-engineering-blogs) -* [Under development](#under-development) -* [Credits](#credits) -* [Contact info](#contact-info) -* [License](#license) +- [The System Design Primer](#the-system-design-primer) + - [Motivation](#motivation) + - [Learn how to design large-scale systems](#learn-how-to-design-large-scale-systems) + - [Learn from the open source community](#learn-from-the-open-source-community) + - [Prep for the system design interview](#prep-for-the-system-design-interview) + - [Anki flashcards](#anki-flashcards) + - [Coding Resource: Interactive Coding Challenges](#coding-resource-interactive-coding-challenges) + - [Contributing](#contributing) + - [Index of system design topics](#index-of-system-design-topics) + - [Study guide](#study-guide) + - [How to approach a system design interview question](#how-to-approach-a-system-design-interview-question) + - [Step 1: Outline use cases, constraints, and assumptions](#step-1-outline-use-cases-constraints-and-assumptions) + - [Step 2: Create a high level design](#step-2-create-a-high-level-design) + - [Step 3: Design core components](#step-3-design-core-components) + - [Step 4: Scale the design](#step-4-scale-the-design) + - [Back-of-the-envelope calculations](#back-of-the-envelope-calculations) + - [Source(s) and further reading](#sources-and-further-reading) + - [System design interview questions with solutions](#system-design-interview-questions-with-solutions) + - [Design Pastebin.com (or Bit.ly)](#design-pastebincom-or-bitly) + - [Design the Twitter timeline and search (or Facebook feed and search)](#design-the-twitter-timeline-and-search-or-facebook-feed-and-search) + - [Design a web crawler](#design-a-web-crawler) + - [Design Mint.com](#design-mintcom) + - [Design the data structures for a social network](#design-the-data-structures-for-a-social-network) + - [Design a key-value store for a search engine](#design-a-key-value-store-for-a-search-engine) + - [Design Amazon's sales ranking by category feature](#design-amazons-sales-ranking-by-category-feature) + - [Design a system that scales to millions of users on AWS](#design-a-system-that-scales-to-millions-of-users-on-aws) + - [Object-oriented design interview questions with solutions](#object-oriented-design-interview-questions-with-solutions) + - [System design topics: start here](#system-design-topics-start-here) + - [Step 1: Review the scalability video lecture](#step-1-review-the-scalability-video-lecture) + - [Step 2: Review the scalability article](#step-2-review-the-scalability-article) + - [Next steps](#next-steps) + - [Performance vs scalability](#performance-vs-scalability) + - [Source(s) and further reading](#sources-and-further-reading-1) + - [Latency vs throughput](#latency-vs-throughput) + - [Source(s) and further reading](#sources-and-further-reading-2) + - [Availability vs consistency](#availability-vs-consistency) + - [CAP theorem](#cap-theorem) + - [CP - consistency and partition tolerance](#cp---consistency-and-partition-tolerance) + - [AP - availability and partition tolerance](#ap---availability-and-partition-tolerance) + - [Source(s) and further reading](#sources-and-further-reading-3) + - [Consistency patterns](#consistency-patterns) + - [Weak consistency](#weak-consistency) + - [Eventual consistency](#eventual-consistency) + - [Strong consistency](#strong-consistency) + - [Source(s) and further reading](#sources-and-further-reading-4) + - [Availability patterns](#availability-patterns) + - [Fail-over](#fail-over) + - [Active-passive](#active-passive) + - [Active-active](#active-active) + - [Disadvantage(s): failover](#disadvantages-failover) + - [Replication](#replication) + - [Master-slave and master-master](#master-slave-and-master-master) + - [Availability in numbers](#availability-in-numbers) + - [99.9% availability - three 9s](#999-availability---three-9s) + - [99.99% availability - four 9s](#9999-availability---four-9s) + - [Availability in parallel vs in sequence](#availability-in-parallel-vs-in-sequence) + - [In sequence](#in-sequence) + - [In parallel](#in-parallel) + - [Domain name system](#domain-name-system) + - [Disadvantage(s): DNS](#disadvantages-dns) + - [Source(s) and further reading](#sources-and-further-reading-5) + - [Content delivery network](#content-delivery-network) + - [Push CDNs](#push-cdns) + - [Pull CDNs](#pull-cdns) + - [Disadvantage(s): CDN](#disadvantages-cdn) + - [Source(s) and further reading](#sources-and-further-reading-6) + - [Load balancer](#load-balancer) + - [Layer 4 load balancing](#layer-4-load-balancing) + - [Layer 7 load balancing](#layer-7-load-balancing) + - [Horizontal scaling](#horizontal-scaling) + - [Disadvantage(s): horizontal scaling](#disadvantages-horizontal-scaling) + - [Disadvantage(s): load balancer](#disadvantages-load-balancer) + - [Source(s) and further reading](#sources-and-further-reading-7) + - [Reverse proxy (web server)](#reverse-proxy-web-server) + - [Load balancer vs reverse proxy](#load-balancer-vs-reverse-proxy) + - [Disadvantage(s): reverse proxy](#disadvantages-reverse-proxy) + - [Source(s) and further reading](#sources-and-further-reading-8) + - [Application layer](#application-layer) + - [Microservices](#microservices) + - [Service Discovery](#service-discovery) + - [Disadvantage(s): application layer](#disadvantages-application-layer) + - [Source(s) and further reading](#sources-and-further-reading-9) + - [Database](#database) + - [Relational database management system (RDBMS)](#relational-database-management-system-rdbms) + - [Master-slave replication](#master-slave-replication) + - [Disadvantage(s): master-slave replication](#disadvantages-master-slave-replication) + - [Master-master replication](#master-master-replication) + - [Disadvantage(s): master-master replication](#disadvantages-master-master-replication) + - [Disadvantage(s): replication](#disadvantages-replication) + - [Source(s) and further reading: replication](#sources-and-further-reading-replication) + - [Federation](#federation) + - [Disadvantage(s): federation](#disadvantages-federation) + - [Source(s) and further reading: federation](#sources-and-further-reading-federation) + - [Sharding](#sharding) + - [Disadvantage(s): sharding](#disadvantages-sharding) + - [Source(s) and further reading: sharding](#sources-and-further-reading-sharding) + - [Denormalization](#denormalization) + - [Disadvantage(s): denormalization](#disadvantages-denormalization) + - [Source(s) and further reading: denormalization](#sources-and-further-reading-denormalization) + - [SQL tuning](#sql-tuning) + - [Tighten up the schema](#tighten-up-the-schema) + - [Use good indices](#use-good-indices) + - [Avoid expensive joins](#avoid-expensive-joins) + - [Partition tables](#partition-tables) + - [Tune the query cache](#tune-the-query-cache) + - [Source(s) and further reading: SQL tuning](#sources-and-further-reading-sql-tuning) + - [NoSQL](#nosql) + - [Key-value store](#key-value-store) + - [Source(s) and further reading: key-value store](#sources-and-further-reading-key-value-store) + - [Document store](#document-store) + - [Source(s) and further reading: document store](#sources-and-further-reading-document-store) + - [Wide column store](#wide-column-store) + - [Source(s) and further reading: wide column store](#sources-and-further-reading-wide-column-store) + - [Graph database](#graph-database) + - [Source(s) and further reading: graph](#sources-and-further-reading-graph) + - [Source(s) and further reading: NoSQL](#sources-and-further-reading-nosql) + - [SQL or NoSQL](#sql-or-nosql) + - [Source(s) and further reading: SQL or NoSQL](#sources-and-further-reading-sql-or-nosql) + - [Cache](#cache) + - [Client caching](#client-caching) + - [CDN caching](#cdn-caching) + - [Web server caching](#web-server-caching) + - [Database caching](#database-caching) + - [Application caching](#application-caching) + - [Caching at the database query level](#caching-at-the-database-query-level) + - [Caching at the object level](#caching-at-the-object-level) + - [When to update the cache](#when-to-update-the-cache) + - [Cache-aside](#cache-aside) + - [Disadvantage(s): cache-aside](#disadvantages-cache-aside) + - [Write-through](#write-through) + - [Disadvantage(s): write through](#disadvantages-write-through) + - [Write-behind (write-back)](#write-behind-write-back) + - [Disadvantage(s): write-behind](#disadvantages-write-behind) + - [Refresh-ahead](#refresh-ahead) + - [Disadvantage(s): refresh-ahead](#disadvantages-refresh-ahead) + - [Disadvantage(s): cache](#disadvantages-cache) + - [Source(s) and further reading](#sources-and-further-reading-10) + - [Asynchronism](#asynchronism) + - [Message queues](#message-queues) + - [Task queues](#task-queues) + - [Back pressure](#back-pressure) + - [Disadvantage(s): asynchronism](#disadvantages-asynchronism) + - [Source(s) and further reading](#sources-and-further-reading-11) + - [Communication](#communication) + - [Hypertext transfer protocol (HTTP)](#hypertext-transfer-protocol-http) + - [Source(s) and further reading: HTTP](#sources-and-further-reading-http) + - [Transmission control protocol (TCP)](#transmission-control-protocol-tcp) + - [User datagram protocol (UDP)](#user-datagram-protocol-udp) + - [Source(s) and further reading: TCP and UDP](#sources-and-further-reading-tcp-and-udp) + - [Remote procedure call (RPC)](#remote-procedure-call-rpc) + - [Disadvantage(s): RPC](#disadvantages-rpc) + - [Representational state transfer (REST)](#representational-state-transfer-rest) + - [Disadvantage(s): REST](#disadvantages-rest) + - [RPC and REST calls comparison](#rpc-and-rest-calls-comparison) + - [Source(s) and further reading: REST and RPC](#sources-and-further-reading-rest-and-rpc) + - [Security](#security) + - [Source(s) and further reading](#sources-and-further-reading-12) + - [Appendix](#appendix) + - [Powers of two table](#powers-of-two-table) + - [Source(s) and further reading](#sources-and-further-reading-13) + - [Latency numbers every programmer should know](#latency-numbers-every-programmer-should-know) + - [Latency numbers visualized](#latency-numbers-visualized) + - [Source(s) and further reading](#sources-and-further-reading-14) + - [Additional system design interview questions](#additional-system-design-interview-questions) + - [Real world architectures](#real-world-architectures) + - [Company architectures](#company-architectures) + - [Company engineering blogs](#company-engineering-blogs) + - [Source(s) and further reading](#sources-and-further-reading-15) + - [Under development](#under-development) + - [Credits](#credits) + - [Contact info](#contact-info) + - [License](#license) ## Study guide @@ -608,7 +699,7 @@ Services such as [CloudFlare](https://www.cloudflare.com/dns/) and [Route 53](ht * Accessing a DNS server introduces a slight delay, although mitigated by caching described above. * DNS server management could be complex and is generally managed by [governments, ISPs, and large companies](http://superuser.com/questions/472695/who-controls-the-dns-servers/472729). -* DNS services have recently come under [DDoS attack](http://dyn.com/blog/dyn-analysis-summary-of-friday-october-21-attack/), preventing users from accessing websites such as Twitter without knowing Twitter's IP address(es). +* DNS services have recently come under [DDoS attack](http://dyn.com/blog/dyn-analysis-summary-of-friday-october-21-attack/), preventing users from accessing websites such as Twitter without knowing Twitter's IP address(es). Common techniques to prevent DDoS are implementing Rate limiting and Firewall. ### Source(s) and further reading