From 7df014511809c1a91edebcde032eb025868471ac Mon Sep 17 00:00:00 2001 From: Jimmy Zelinskie Date: Fri, 31 Aug 2018 00:56:18 -0400 Subject: [PATCH 1/2] readme: refresh info, add graphviz architecture --- README.md | 75 +++++++++++++++---------------------------- docs/architecture.dot | 36 +++++++++++++++++++++ 2 files changed, 62 insertions(+), 49 deletions(-) create mode 100644 docs/architecture.dot diff --git a/README.md b/README.md index 39faa59..182ca8b 100644 --- a/README.md +++ b/README.md @@ -15,49 +15,62 @@ Chihaya is an open source [BitTorrent tracker] written in [Go]. Differentiating features include: -- Protocol-agnostic middleware -- HTTP and UDP frontends +- HTTP and UDP protocols - IPv4 and IPv6 support +- Pre/Post middlware hooks - [YAML] configuration - Metrics via [Prometheus] - High Availability via [Redis] +- Kubernetes deployment via [Helm] [releases]: https://github.com/chihaya/chihaya/releases -[BitTorrent tracker]: http://en.wikipedia.org/wiki/BitTorrent_tracker +[BitTorrent tracker]: https://en.wikipedia.org/wiki/BitTorrent_tracker [Go]: https://golang.org -[YAML]: http://yaml.org -[Prometheus]: http://prometheus.io +[YAML]: https://yaml.org +[Prometheus]: https://prometheus.io [Redis]: https://redis.io +[Helm]: https://helm.sh ## Why Chihaya? Chihaya is built for developers looking to integrate BitTorrent into a preexisting production environment. Chihaya's pluggable architecture and middleware framework offers a simple and flexible integration point that abstracts the BitTorrent tracker protocols. -The most common use case for Chihaya is integration with the deployment of cloud software. - -[OpenBittorrent]: https://openbittorrent.com +The most common use case for Chihaya is enabling peer-to-peer cloud software deployments. ### Production Use #### Facebook [Facebook] uses BitTorrent to deploy new versions of their software. -In order to optimize the flow of traffic within their datacenters, Chihaya is configured to prefer peers within the same subnet. +In order to optimize the flow of traffic within their datacenters, Facebook uses an alternative Storage driver for Chihaya that is configured to prefer peers within the same subnet. Because Facebook organizes their network such that server racks are allocated IP addresses in the same subnet, the vast majority of deployment traffic never impacts the congested areas of their network. [Facebook]: https://facebook.com -#### CoreOS +#### Red Hat [Quay] is a container registry that offers the ability to download containers via BitTorrent in order to speed up large or geographically distant deployments. Announce URLs from Quay's torrent files contain a [JWT] in order to allow Chihaya to verify that an infohash was approved by the registry. -By verifying the infohash, Quay can be sure that only their content is being shared by their tracker. +By verifying the infohash, Quay can restrict their tracker to only sharing their own content. [Quay]: https://quay.io [JWT]: https://jwt.io ## Development +### Contributing + +Long-term discussion and bug reports are maintained via [GitHub Issues]. +Code review is done via [GitHub Pull Requests]. +Real-time discussion is done via [freenode IRC]. + +For more information read [CONTRIBUTING.md]. + +[GitHub Issues]: https://github.com/chihaya/chihaya/issues +[GitHub Pull Requests]: https://github.com/chihaya/chihaya/pulls +[freenode IRC]: http://webchat.freenode.net/?channels=chihaya +[CONTRIBUTING.md]: https://github.com/chihaya/chihaya/blob/master/CONTRIBUTING.md + ### Getting Started #### Building from HEAD @@ -108,45 +121,9 @@ Removing `-bench` will just run unit tests. $ go test -bench $(go list ./...) ``` -### Contributing - -Long-term discussion and bug reports are maintained via [GitHub Issues]. -Code review is done via [GitHub Pull Requests]. -Real-time discussion is done via [freenode IRC]. - -For more information read [CONTRIBUTING.md]. - -[GitHub Issues]: https://github.com/chihaya/chihaya/issues -[GitHub Pull Requests]: https://github.com/chihaya/chihaya/pulls -[freenode IRC]: http://webchat.freenode.net/?channels=chihaya -[CONTRIBUTING.md]: https://github.com/chihaya/chihaya/blob/master/CONTRIBUTING.md - ### Architecture -``` - +----------------------+ - | BitTorrent Client |<--------------+ - +----------------------+ | - | | - | | - | | -+------------v--------------------------+-------------------+-------------------------+ -|+----------------------+ +----------------------+frontend| chihaya| -|| Parser | | Writer | | | -|+----------------------+ +----------------------+ | | -| | ^ | | -+------------+--------------------------+-------------------+ | -+------------v--------------------------+-------------------+ | -|+----------------------+ +----------------------+ logic| | -|| PreHook Middleware |-->| Response Generator |<-------|-------------+ | -|+----------------------+ +----------------------+ | | | -| | | | -|+----------------------+ | +----------------------+| -|| PostHook Middleware |-----------------------------------|>| Storage || -|+----------------------+ | +----------------------+| -| | | -+-----------------------------------------------------------+-------------------------+ -``` +![](https://user-images.githubusercontent.com/343539/52676700-05c45c80-2ef9-11e9-9887-8366008b4e7e.png) BitTorrent clients send Announce and Scrape requests to a _Frontend_. Frontends parse requests and write responses for the particular protocol they implement. @@ -155,7 +132,7 @@ A configurable chain of _PreHook_ and _PostHook_ middleware is used to construct PreHooks are middleware that are executed before the response has been written. After all PreHooks have executed, any missing response fields that are required are filled by reading out of the configured implementation of the _Storage_ interface. PostHooks are asynchronous tasks that occur after a response has been delivered to the client. -Request data is written to the storage asynchronously in one of these PostHooks. +Because they are unnecessary to for generating a response, updates to the Storage for a particular request are done asynchronously in a PostHook. ## Related projects diff --git a/docs/architecture.dot b/docs/architecture.dot new file mode 100644 index 0000000..f1233c9 --- /dev/null +++ b/docs/architecture.dot @@ -0,0 +1,36 @@ +digraph G { + subgraph cluster_0 { + label = "chihaya"; + style = "line"; + color = "blue"; + + "Storage"; + + subgraph cluster_1 { + label = "frontend"; + style = "line"; + color = "hotpink"; + + "Parser"; + "Writer"; + } + + subgraph cluster_2 { + label = "logic"; + style = "line"; + color = "purple"; + + "PreHook Middleware"; + "PostHook Middleware"; + "Response Generator"; + } + } + + "BitTorrent Client" -> "Parser"; + "Parser" -> "PreHook Middleware"; + "PreHook Middleware" -> "Response Generator"; + "PostHook Middleware" -> "Storage"; + "Storage" -> "Response Generator"; + "Response Generator" -> "Writer"; + "Writer" -> "BitTorrent Client"; +} From 6e362c184c2dc1a832f4624a0b96ed55600eaae6 Mon Sep 17 00:00:00 2001 From: Jimmy Zelinskie Date: Tue, 12 Feb 2019 19:23:30 -0500 Subject: [PATCH 2/2] docs: refactor architecture into its own doc --- README.md | 13 ------------- docs/architecture.md | 16 ++++++++++++++++ 2 files changed, 16 insertions(+), 13 deletions(-) create mode 100644 docs/architecture.md diff --git a/README.md b/README.md index 182ca8b..982e80f 100644 --- a/README.md +++ b/README.md @@ -121,19 +121,6 @@ Removing `-bench` will just run unit tests. $ go test -bench $(go list ./...) ``` -### Architecture - -![](https://user-images.githubusercontent.com/343539/52676700-05c45c80-2ef9-11e9-9887-8366008b4e7e.png) - -BitTorrent clients send Announce and Scrape requests to a _Frontend_. -Frontends parse requests and write responses for the particular protocol they implement. -The _TrackerLogic_ interface to is used to generate responses for their requests and optionally perform a task after responding to a client. -A configurable chain of _PreHook_ and _PostHook_ middleware is used to construct an instance of TrackerLogic. -PreHooks are middleware that are executed before the response has been written. -After all PreHooks have executed, any missing response fields that are required are filled by reading out of the configured implementation of the _Storage_ interface. -PostHooks are asynchronous tasks that occur after a response has been delivered to the client. -Because they are unnecessary to for generating a response, updates to the Storage for a particular request are done asynchronously in a PostHook. - ## Related projects - [BitTorrent.org](https://github.com/bittorrent/bittorrent.org): a static website containing the BitTorrent spec and all BEPs diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..dad5b8e --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,16 @@ +# Architecture + +### Overview + +BitTorrent clients send Announce and Scrape requests to a _Frontend_. +Frontends parse requests and write responses for the particular protocol they implement. +The _TrackerLogic_ interface is used to generate responses for requests and optionally perform a task after responding to a client. +A configurable chain of _PreHook_ and _PostHook_ middleware is used to construct an instance of TrackerLogic. +PreHooks are middleware that are executed before the response has been written. +After all PreHooks have executed, any missing response fields that are required are filled by reading out of the configured implementation of the _Storage_ interface. +PostHooks are asynchronous tasks that occur after a response has been delivered to the client. +Because they are unnecessary to for generating a response, updates to the Storage for a particular request are done asynchronously in a PostHook. + +### Diagram + +![](https://user-images.githubusercontent.com/343539/52676700-05c45c80-2ef9-11e9-9887-8366008b4e7e.png)