Fill protocol pages
This commit is contained in:
Ben van Hartingsveldt
parent
35d0ef20b3
commit
0b302314a5
3 changed files with 205 additions and 0 deletions
|
|
@ -2,3 +2,59 @@
|
|||
title: Content Protocol
|
||||
description: The protocol used to download blobs.
|
||||
---
|
||||
|
||||
This protocol provides a way to pull a blob from a specific node. The protocol uses JSON blocks. The end of a JSON block can be detected if the JSON is valid after receiving a `}`, thereby closing the root object. Another JSON block directly follows it, without any whitespace. This protocol supports composite requests, so multiple requests can be merged into one, as also for the response. For example, one request can both contain `requested_blobs` and `blob_data_payment_rate`, and the server will answer with one response containing both `available_blobs` and `blob_data_payment_rate`.
|
||||
|
||||
- **Port:** 3333/TCP (distributed peer, previously)
|
||||
- **Port:** 4444/TCP (distributed peer)
|
||||
- **Port:** 5567/TCP (fixed peer)
|
||||
|
||||
### Availability
|
||||
|
||||
This request will check what blobs are available on the peer. It takes the sent list of blob hashes from `requested_blobs`, checks everyone of them, and returns a list of available blobs in `available_blobs`. Also, the request contains `lbrycrd_address`, which is a boolean property. In the response the `lbrycrd_address` property is a string containing an address. However, the use of `lbrycrd_address` for paying for blob data is not implemented yet.
|
||||
|
||||
```json5
|
||||
//REQUEST
|
||||
{"lbrycrd_address":false,"requested_blobs":["aabbcc","ddeeff"]}
|
||||
//or
|
||||
{"lbrycrd_address":true,"requested_blobs":["aabbcc","ddeeff"]}
|
||||
|
||||
//RESPONSE
|
||||
{"lbrycrd_address":"bJxKvpD96kaJLriqVajZ7SaQTsWWyrGQct","requested_blobs":["aabbcc","ddeeff"]}
|
||||
//or
|
||||
{"lbrycrd_address":"bJxKvpD96kaJLriqVajZ7SaQTsWWyrGQct","requested_blobs":["aabbcc"]}
|
||||
//or
|
||||
{"lbrycrd_address":"bJxKvpD96kaJLriqVajZ7SaQTsWWyrGQct","requested_blobs":["ddeeff"]}
|
||||
//or
|
||||
{"lbrycrd_address":"bJxKvpD96kaJLriqVajZ7SaQTsWWyrGQct","requested_blobs":[]}
|
||||
```
|
||||
|
||||
### Payment Rate
|
||||
|
||||
The reason behind this request is unknown at this moment. The request contains the `blob_data_payment_rate` property with a float. The result is always `RATE_ACCEPTED`, unless the float is below zero. Then the server will respond with `RATE_TOO_LOW`. At this moment, `RATE_UNSET` is not used.
|
||||
|
||||
```json5
|
||||
//REQUEST
|
||||
{"blob_data_payment_rate":123.456}
|
||||
|
||||
//RESPONSE
|
||||
{"blob_data_payment_rate":"RATE_ACCEPTED"}
|
||||
//or
|
||||
{"blob_data_payment_rate":"RATE_TOO_LOW"}
|
||||
//or
|
||||
{"blob_data_payment_rate":"RATE_UNSET"}
|
||||
```
|
||||
|
||||
### Blob
|
||||
|
||||
This request will get the blob itself. The hash will be sent using `requested_blob`. The server then reacts with an `incoming_blob`. If everything is correct, the `incoming_blob` contains a `blob_hash` property and a `length` property. The blob data directly follows after the JSON block response. If an error occurs, the `incoming_blob` contains an `error` property (and `blob_hash` is empty and `length` is zero).
|
||||
|
||||
```json5
|
||||
//REQUEST
|
||||
{"requested_blob":"aabbcc"}
|
||||
|
||||
//RESPONSE
|
||||
{"incoming_blob":{"blob_hash":"aabbcc","length":123}}/*[Raw Blob Data]*/
|
||||
//or
|
||||
{"incoming_blob":{"blob_hash":"","length":0,"error":"Blob not found"}}
|
||||
```
|
||||
|
|
|
|||
|
|
@ -2,3 +2,84 @@
|
|||
title: DHT Protocol
|
||||
description: The protocol used to find other nodes.
|
||||
---
|
||||
|
||||
The DHT protocol is used to discover and connect with other nodes. It follows the Kademlia specification, with some slight modiciations. In the Kademlia spec there are 4 message types: `PING`, `FIND_NODE`, `FIND_VALUE` and `STORE`. The protocol uses Bencoding, but is different from the BitTorrent specification (BEP 5).
|
||||
|
||||
- **Port:** 4444/UDP
|
||||
|
||||
The root dictionary can hold 5 properties: `0`, `1`, `2`, `3` and `4`. Note: Bencoding requires those keys to be strings, but some old software could send those as integers. This is wrong. The request must have all those properties, the response only requires `0`, `1`, `2` and `3`. The error, like the request, also requires all properties.
|
||||
|
||||
- The `0` property is the message type, where the value is an integer with `0` for request, `1` for response and `2` for error.
|
||||
- The `1` property is the message ID, which contains a string of 20 bytes and is unique for every request-response or request-error pair.
|
||||
- The `2` property is the node ID, which contains a string of 48 bytes and is unique for every node.
|
||||
- The `3` property is the method (string; one of the 4) in case of a request message type, the response (mixed) in case of a response message type, and the error type (string) in case of a error message type.
|
||||
- The `4` property is the arguments (list) in case of a request message type, absent in case of a response message type, and the response (string) in case of a error message type.
|
||||
|
||||
When a node wants to send an error back (it doesn't need to in every case), then the error message looks like the response message. However, the `3` property now contains the error type and optionally, `4` contains more information.
|
||||
|
||||
### Ping
|
||||
|
||||
The request has a property `3` with value `ping`.
|
||||
|
||||
In protocol version `0`, there are no request arguments.
|
||||
|
||||
In protocol version `1`, there is 1 request argument:
|
||||
- A dictionary containing `protocolVersion` with integer value `1`.
|
||||
|
||||
In both versions, the response has a property `3` with value `pong`.
|
||||
|
||||
### Find Node
|
||||
|
||||
The request has a property `3` with value `findNode`.
|
||||
|
||||
In protocol version `0`, there is 1 request argument:
|
||||
- A 48-byte string containing a key.
|
||||
|
||||
In protocol version `1`, there are 2 request arguments:
|
||||
- A 48-byte string containing a key.
|
||||
- A dictionary containing `protocolVersion` with integer value `1`.
|
||||
|
||||
In both versions, the response has a property `3` with a list as value. Every item in the list is a tuple (another list), containing the node ID as 48-byte string, the IP as string and the port as integer.
|
||||
|
||||
### Find Value
|
||||
|
||||
The request has a property `3` with value `findValue`.
|
||||
|
||||
In protocol version `0`, there is 1 request argument:
|
||||
- A 48-byte string containing a key.
|
||||
|
||||
In protocol version `1`, there are 2 request arguments:
|
||||
- A 48-byte string containing a key.
|
||||
- A dictionary containing `p` (optionally) with an integer value to indicate the page, and `protocolVersion` with integer value `1`.
|
||||
|
||||
In both versions, the response has a property `3` with a dictionary as value:
|
||||
- The directory at least contains a property `token`, which is needed when storing values on the connected node. If the node supports protocol version `1`, it should have `protocolVersion` set to integer `1`. If it only supports version `0`, `protocolVersion` can be integer `0` or absent.
|
||||
- Based on if the node has the key, it will return the `contacts` property or the property where the name is the same as the key argument in the request. If the key is not known by the node, the `contacts` property is present. Like the `findNode` function response, the `contacts` property will contain a list of tuples, containing the node ID, IP address and DHT port number.
|
||||
- If the key is known by the node, `contacts` isn't required, but allowed to be present. The response now at least has a property with the same 48 byte long name as the key in the request. This property is a list, where every item in the list is a compact address. This compact address contains information on where the blob can be downloaded.
|
||||
- The optional `p` property is an integer with the amount of pages of download locations. If there are no download locations, the `p` value is integer `0`. If the `p` property is set in the request, the `contacts` property is absent.
|
||||
|
||||
#### Compact Address
|
||||
|
||||
The compact address is a value of 54 bytes. The first 4 bytes are the binary format of the IPv4 address. The next 2 bytes are the TCP port where the blob can be downloaded. The remanining 48 bytes are the node id of the associated DHT peer.
|
||||
|
||||
### Store
|
||||
|
||||
The request has a property `3` with value `store`.
|
||||
|
||||
In protocol version `0`, there are 4 request arguments:
|
||||
- A 48-byte string containing a key (e.g. blob hash).
|
||||
- A value.
|
||||
- An original publisher ID.
|
||||
- An age.
|
||||
|
||||
The value is a dictory with the properties `token`, `lbryid` and `port`. The `token` property holds the token value of the connected node, where the token value is received from an earlier `findValue` request. The `lbryid` property holds the node ID of the sending node. The `port` property holds the port number of where the blob can be downloaded.
|
||||
|
||||
In protocol version `1`, there are 6 request arguments:
|
||||
- A blob hash.
|
||||
- A token.
|
||||
- A port.
|
||||
- An original publisher ID.
|
||||
- An age.
|
||||
- A dictionary containing `protocolVersion` with integer value `1`.
|
||||
|
||||
In both versions, the response has a property `3` with value `OK`.
|
||||
|
|
|
|||
|
|
@ -2,3 +2,71 @@
|
|||
title: Reflector Protocol
|
||||
description: The protocol used to upload blobs.
|
||||
---
|
||||
|
||||
This protocol provides a way to actively push blobs to the network, instead of waiting for them to get pulled. The protocol uses JSON blocks. The end of a JSON block can be detected if the JSON is valid after receiving a `}`, thereby closing the root object. Another JSON block directly follows it, without any whitespace. At this moment, there will be no error block sent when an error occurs. Version `0` of the protocol supports blobs, version `1` of the protocol also supports SD blobs. Unknown properties are ignored and will not cause an error.
|
||||
|
||||
- **Port:** 5566/TCP
|
||||
|
||||
### Handshake
|
||||
|
||||
Before doing anything with blobs, the handshake should be sent. The `version` property is required and should be an integer valued `0` or `1`. If the server supports that version, it will send back an JSON block where the `version` property has the same value. If the `value` property contains an invalid protocol number or a protocol number that isn't supported, the server throws an error and closes the connection.
|
||||
|
||||
```json5
|
||||
//REQUEST
|
||||
{"version":0}
|
||||
//or
|
||||
{"version":1}
|
||||
|
||||
//RESPONSE
|
||||
{"version":0}
|
||||
//or
|
||||
{"version":1}
|
||||
```
|
||||
|
||||
### Blob Hash and Blob Size
|
||||
|
||||
After the handshake, blobs can be received. First, the server wants to know more about the blob itself, before it wants to receive the blob data itself. The clients needs to send a JSON block with the blob hash and the blob size. It is important to note that this request is different for normal blobs and SD (Stream Descriptor) blobs. The properties `blob_hash` and `blob_size` are required OR the properties `sd_blob_hash` and `sd_blob_size` are required. The hash cannot be empty and the size cannot be zero or exceed the maximum blob size.
|
||||
|
||||
Then, the server will check if it wants to receive the blob. It can do several checks, e.g. checking if it already has the blob or if the blob is blacklisted. If the server wants the blob, it will send a JSON block with the `send_blob` property for normal blobs and a JSON block with `send_sd_blob` for SD blobs. If the server notices that it has the SD blob, but misses some of the blobs defined in the SD blob, it will at those hashes to `needed_blobs`. If the server doesn't want the blob, the client can send information about another blob.
|
||||
|
||||
```json5
|
||||
//REQUEST
|
||||
{"blob_hash":"aabbcc","blob_size":123} // if version>=0
|
||||
//or
|
||||
{"sd_blob_hash":"ddeeff","sd_blob_size":456} // if version>=1
|
||||
|
||||
//RESPONSE
|
||||
{"send_blob":false} // if version>=0
|
||||
//or
|
||||
{"send_blob":true} // if version>=0
|
||||
//or
|
||||
{"send_sd_blob":false} // if version>=1
|
||||
//or
|
||||
{"send_sd_blob":true} // if version>=1
|
||||
//or
|
||||
{"send_sd_blob":false,"needed_blobs":[]} // if version>=1
|
||||
//or
|
||||
{"send_sd_blob":true,"needed_blobs":[]} // if version>=1
|
||||
//or
|
||||
{"send_sd_blob":false,"needed_blobs":["aabbcc","ddeeff"]} // if version>=1
|
||||
//or
|
||||
{"send_sd_blob":true,"needed_blobs":["aabbcc","ddeeff"]} // if version>=1
|
||||
```
|
||||
|
||||
### Blob Data
|
||||
|
||||
If the server wants the blob, it will read exactly the amount of bytes that were stated in the blob information. If there goes something wrong with sending the blob data (e.g. there was a socket timeout or the blob hash calculation didn't match the blob hash from the blob information), the server will send a message that it didn't receive the blob. This will be a JSON block with the `received_blob` property for normal blobs and a JSON block with `received_sd_blob` for SD blobs. After this message, the client can send information about another blob.
|
||||
|
||||
```json5
|
||||
//REQUEST
|
||||
/*[Raw Blob Data]*/
|
||||
|
||||
//RESPONSE
|
||||
{"received_blob":false} // if version>=0
|
||||
//or
|
||||
{"received_blob":true} // if version>=0
|
||||
//or
|
||||
{"received_sd_blob":false} // if version>=1
|
||||
//or
|
||||
{"received_sd_blob":true} // if version>=1
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue