Pigges/lbry-tech
1
0
Fork 0
mirror of https://github.com/LBRYFoundation/lbry-tech.git synced 2025-03-04 06:57:53 +00:00
Code Issues Projects Releases Packages Wiki Activity

create collection list and toc

Browse source
This commit is contained in:
Philip Ahlqvist 2023-12-05 22:19:20 +01:00
parent 370610d524
commit fd058b6582
27 changed files with 1462 additions and 56 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

30
src/components/TableOfContents.astro Normal file
View file

@ -0,0 +1,30 @@
---
// TableOfContents.astro
import TableOfContentsHeading from "./TableOfContentsHeading.astro";
const { headings } = Astro.props;
const toc = headings && headings.length ? buildToc(headings) : [];
function buildToc(headings) {
const toc = [];
const parentHeadings = new Map();
headings.forEach((h) => {
const heading = { ...h, subheadings: [] };
parentHeadings.set(heading.depth, heading);
// Change 2 to 1 if your markdown includes your <h1>
// if (heading.depth === 2) {
// toc.push(heading);
// } else {
// parentHeadings.get(heading.depth - 1).subheadings.push(heading);
// }
toc.push(heading);
});
return toc;
}
---
<nav class="toc">
<ul>
{toc.map((heading) => <TableOfContentsHeading heading={heading} />)}
</ul>
</nav>

19
src/components/TableOfContentsHeading.astro Normal file
View file

@ -0,0 +1,19 @@
---
// TableOfContentsHeading.astro
const { heading } = Astro.props;
---
<li>
<a href={'#' + heading.slug}>
{heading.text}
</a>
{
heading.subheadings.length > 0 && (
<ul>
{heading.subheadings.map((subheading) => (
<Astro.self heading={subheading} />
))}
</ul>
)
}
</li>

151
src/content/resources/android-build.mdx Normal file
View file

@ -0,0 +1,151 @@
---
title: Building the LBRY Android App
description: Step-by-step build instructions for the LBRY Android App
---
### Introduction
This guide provides step-by-step instructions to setup and build the [LBRY Android App](https://lbry.com/android) for development purposes.
#### Estimated Time
25 - 40 minutes
#### Prerequisites
- A computer running Linux with `apt` (otherwise, alter the package installation instructions accordingly)
- At least 15GB of free disk space
- Internet access
### Environment Setup
You can set up your environment via Docker or manually.
#### Docker Environment Setup
Use [Docker](https://docs.docker.com/install/) and start a container using the `lbry/android-base` image. Run:
```bash
docker run -it lbry/android-base:latest /bin/bash
```
When this is complete, continue to [Building the App](#building-the-app).
#### Manual Environment Setup
##### 1. Install Packages via Apt
Install required system packages via `apt`:
```bash
sudo dpkg --add-architecture i386
sudo apt-get -y update
sudo apt-get install -y curl ca-certificates software-properties-common gpg-agent wget
sudo add-apt-repository ppa:deadsnakes/ppa -y && \
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get -y update && apt-get -y install autoconf autogen automake libtool libffi-dev \
build-essential python3.7 python3.7-dev python3.7-venv python3-pip ccache git libncurses5:i386 libstdc++6:i386 \
libgtk2.0-0:i386 libpangox-1.0-0:i386 libpangoxft-1.0-0:i386 libidn11:i386 python2.7 python2.7-dev \
python-pip openjdk-8-jdk unzip zlib1g-dev zlib1g:i386 m4 libc6-dev-i386 yarn gawk nodejs npm
```
##### 2. Setup Python
Install required Python packages:
```bash
sudo -H pip install --upgrade cython==0.28.1 setuptools
```
##### 3. Setup Buildozer
Install buildozer, a tool for creating the apk package using the python for android toolchain:
```bash
git clone https://github.com/lbryio/buildozer.git
cd buildozer && python2.7 setup.py install && cd ..
```
##### 4. Setup Android SDK
The Android SDK needs to be setup for buildozer. This requires creating a few directories and downloading a number of files. Run the following commands to create the buildozer directory, download the required archives and extract them into their proper destination folders:
```bash
mkdir -p ~/.buildozer/android/platform
wget 'https://dl.google.com/android/android-sdk_r23-linux.tgz' -P ~/.buildozer/android/platform/ && \
wget 'https://dl.google.com/android/repository/platform-28_r06.zip' -P ~/.buildozer/android/platform/ && \
wget 'https://dl.google.com/android/repository/build-tools_r26.0.2-linux.zip' -P ~/.buildozer/android/platform/
tar -xvf ~/.buildozer/android/platform/android-sdk_r23-linux.tgz -C ~/.buildozer/android/platform/ && \
mv ~/.buildozer/android/platform/android-sdk-linux ~/.buildozer/android/platform/android-sdk-23 && \
unzip ~/.buildozer/android/platform/platform-28_r06.zip -d ~/.buildozer/android/platform/android-sdk-23/platforms && \
mv ~/.buildozer/android/platform/android-sdk-23/platforms/android-9 ~/.buildozer/android/platform/android-sdk-23/platforms/android-28 && \
mkdir -p ~/.buildozer/android/platform/android-sdk-23/build-tools && \
unzip ~/.buildozer/android/platform/build-tools_r26.0.2-linux.zip -d ~/.buildozer/android/platform/android-sdk-23/build-tools && \
mkdir -p ~/.buildozer/android/platform/android-sdk-23/licenses && \
echo $'\nd56f5187479451eabf01fb78af6dfcb131a6481e' > ~/.buildozer/android/platform/android-sdk-23/licenses/android-sdk-license
```
##### 5. Install React
Install the react-native-cli npm package:
```bash
sudo npm install -g react-native-cli
```
When this is complete, continue to [Building the App](#building-the-app).
### Building the App
After [preparing your environment](#setup-environment), complete the steps below.
##### 1. Install Crystax
Crystax NDK is required for building Python 3.7 for the mobile app and a number of native C / C++ modules and packages used by the app. Run the following commands to download and extract the NDK:
```bash
wget 'https://www.crystax.net/download/crystax-ndk-10.3.2-linux-x86_64.tar.xz' -P ~/.buildozer/android/ && \
tar -xvf ~/.buildozer/android/crystax-ndk-10.3.2-linux-x86_64.tar.xz -C ~/.buildozer/android/ && \
rm -rf ~/.buildozer/android/crystax-ndk-10.3.2/platforms/android-9 && \
ln -s ~/.buildozer/android/crystax-ndk-10.3.2/platforms/android-21 ~/.buildozer/android/crystax-ndk-10.3.2/platforms/android-9
```
##### 2. Clone and Configure the Repository
Clone the lbryio/lbry-android git repository:
```bash
git clone https://github.com/lbryio/lbry-android
cd lbry-android
cp buildozer.spec.sample buildozer.spec
```
The provided `buildozer.spec.sample` contains defaults provided you followed the environment setup exactly as described. If you altered the steps for your environment or you're encountering build trouble, check `buildozer.spec` to ensure everything is pointing in the right places.
##### 3. Install npm packages
Install the npm packages required for the app's React Native code:
```bash
cd app
npm install
cd ..
```
##### 4. Copy Build Files
Copy required files from the repository for the build to be successful:
```bash
cp scripts/build-target-python.sh ~/.buildozer/android/crystax-ndk-10.3.2/build/tools/build-target-python.sh
cp scripts/mangled-glibc-syscalls.h ~/.buildozer/android/crystax-ndk-10.3.2/platforms/android-21/arch-arm/usr/include/crystax/bionic/libc/include/sys/mangled-glibc-syscalls.h
```
##### 5. Build the thing!
You're finally ready to build the package! You just have to run a single command to generate the APK:
```bash
buildozer android debug
```
### All Set? Now Contribute!
Everything built at LBRY is open source.
Check out [lbry-android](https://github.com/lbryio/lbry-android) on GitHub to find issues or read our [Contributor's Guide](/contribute) to learn more.

132
src/content/resources/claim-signing.md Normal file
View file

@ -0,0 +1,132 @@
---
title: Claim Signing
description: How does signing a claim (a publish) work in LBRY-land? This page explains in great detail. Seriously great detail. The greatest detail anyone has ever seen!
---
Reilly wants to publish _Terror on the Midway_ to the channel he claimed `lbry://@FleischerSuperman`. He picks the name "terroronthemidway", and fills in the information for the claim:
{
"claimType": "streamType",
"stream": {
"metadata": {
"author": "Paramount Pictures",
"description": "The episode in the series of Fleischer Studios-produced Superman serials. Subsequent episodes were produced by Famous Studios.",
"language": "en",
"license": "Public Domain",
"licenseUrl": "",
"nsfw": false,
"preview": "",
"thumbnail": "https://s3.amazonaws.com/files.lbry.io/thumbnails/superman1940-thumb.png",
"title": "Terror on the Midway - Superman Ep 9",
"version": "_0_1_0"
},
"source": {
"contentType": "video/mp4",
"source": "9b70337f51fe9a4481504059b4220ad4f87378d59ecc87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea",
"sourceType": "lbry_sd_hash",
"version": "_0_0_1"
},
"version": "_0_0_1"
},
"version": "_0_0_1"
}
This is serialized as:
080110011ae6020801129e02080410011a24546572726f72206f6e20746865204d6964776179202d2053757065726d616e2045702039227f54686520657069736f6
46520696e2074686520736572696573206f6620466c656973636865722053747564696f732d70726f64756365642053757065726d616e2073657269616c732e2053
756273657175656e7420657069736f64657320776572652070726f64756365642062792046616d6f7573202053747564696f732e2a12506172616d6f756e7420506
9637475726573320d5075626c696320446f6d61696e38004a4868747470733a2f2f73332e616d617a6f6e6177732e636f6d2f66696c65732e6c6272792e696f2f74
68756d626e61696c732f73757065726d616e313934302d7468756d622e706e6752005a001a41080110011a309b70337f51fe9a4481504059b4220ad4f87378d59ec
c87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea2209766964656f2f6d7034
To publish the uri `lbry://@FleischerSuperman/terroronthemidway`, Reilly must sign the SHA256 of the the combined claim address, claim value, and certificate claim id.
He starts by generating an address for the new name claim:
bEGGwBFf39ek6ASJV5YwiUPvDqtpgczCVZ
His claim for the certificate at the uri `lbry://@FleischerSuperman` has a `claim_id` of
2996b9a087c18456402b57cba6085b2a8fcc136d
It contains a der-encoded SECP256k1 public key
{
"certificate": {
"keyType": "SECP256k1",
"publicKey": "3056301006072a8648ce3d020106052b8104000a03420004180488ffcb3d1825af538b0b952f0eba6933faa6d8229609ac0aeadfdbcf49
c59363aa5d77ff2b7ff06cddc07116b335a4a0849b1b524a4a69d908d69f1bcebb",
"version": "_0_0_1"
}
### Signing the claim in detail
Reilly decodes and combines the claim address, the serialized claim value, and the claim id of
the certificate claim. As hex, the combined value is:
5510d538a8ea31210a2a65cfcdd985e3e7bbc42e7c2cecb7bc080110011ae6020801129e02080410011a24546572726f72206f6e20746865204d696477617920
2d2053757065726d616e2045702039227f54686520657069736f646520696e2074686520736572696573206f6620466c656973636865722053747564696f732d
70726f64756365642053757065726d616e2073657269616c732e2053756273657175656e7420657069736f64657320776572652070726f647563656420627920
46616d6f7573202053747564696f732e2a12506172616d6f756e74205069637475726573320d5075626c696320446f6d61696e38004a4868747470733a2f2f73
332e616d617a6f6e6177732e636f6d2f66696c65732e6c6272792e696f2f7468756d626e61696c732f73757065726d616e313934302d7468756d622e706e6752
005a001a41080110011a309b70337f51fe9a4481504059b4220ad4f87378d59ecc87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea2209766964
656f2f6d70342996b9a087c18456402b57cba6085b2a8fcc136d
Then he takes the SHA256 of the combined string, giving:
dea44974ace1893f304cae4073af06a7a6cbb209f97cf8ad5f322216f044304e
He signs (RFC 6979) this hash using the private key to the previously shown certificate, giving the signature:
bf82d53143155bb0cac1fd3d917c000322244b5ad17e7865124db2ed33812ea66c9b0c3f390a65a9e2d452e315e91ae695642847d88e90348ef3c1fa283a36a8
Now he add this signature to the claim:
{
"claimType": "streamType",
"publisherSignature": {
"certificateId": "2996b9a087c18456402b57cba6085b2a8fcc136d",
"signature": "bf82d53143155bb0cac1fd3d917c000322244b5ad17e7865124db2ed33812ea66c9b0c3f390a65a9e2d452e315e91ae695642847d88e90348ef3c1fa283a36a8",
"signatureType": "SECP256k1",
"version": "_0_0_1"
},
"stream": {
"metadata": {
"author": "Paramount Pictures",
"description": "The episode in the series of Fleischer Studios-produced Superman serials. Subsequent episodes were produced by Famous Studios.",
"language": "en",
"license": "Public Domain",
"licenseUrl": "",
"nsfw": false,
"preview": "",
"thumbnail": "https://s3.amazonaws.com/files.lbry.io/thumbnails/superman1940-thumb.png",
"title": "Terror on the Midway - Superman Ep 9",
"version": "_0_1_0"
},
"source": {
"contentType": "video/mp4",
"source": "9b70337f51fe9a4481504059b4220ad4f87378d59ecc87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea",
"sourceType": "lbry_sd_hash",
"version": "_0_0_1"
},
"version": "_0_0_1"
},
"version": "_0_0_1"
}
Serialized, the signed claim is represented as:
080110011ae6020801129e02080410011a24546572726f72206f6e20746865204d6964776179202d2053757065726d616e2045702039227f5468652065706973
6f646520696e2074686520736572696573206f6620466c656973636865722053747564696f732d70726f64756365642053757065726d616e2073657269616c73
2e2053756273657175656e7420657069736f64657320776572652070726f64756365642062792046616d6f7573202053747564696f732e2a12506172616d6f75
6e74205069637475726573320d5075626c696320446f6d61696e38004a4868747470733a2f2f73332e616d617a6f6e6177732e636f6d2f66696c65732e6c6272
792e696f2f7468756d626e61696c732f73757065726d616e313934302d7468756d622e706e6752005a001a41080110011a309b70337f51fe9a4481504059b422
0ad4f87378d59ecc87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea2209766964656f2f6d70342a5c080110031a40bf82d53143155bb0cac1fd
3d917c000322244b5ad17e7865124db2ed33812ea66c9b0c3f390a65a9e2d452e315e91ae695642847d88e90348ef3c1fa283a36a822142996b9a087c1845640
2b57cba6085b2a8fcc136d
Now he can put this value in a claim transaction and broadcast the claim!
As long as his certificate claim is winning, his publication can be resolved by `lbry://@FleischerSuperman/terroronthemidway`
Even if his name is outbid, the publication can be resolved by the channel claim id at the uri: `lbry://@FleischerSuperman#2996b9a087c18456402b57cba6085b2a8fcc136d/terroronthemidway`

74
src/content/resources/claimtrie.md Normal file
View file

@ -0,0 +1,74 @@
---
title: Merkle Claim Trie
description: Publishes to the LBRY network get placed into a Merkle Claim Trie. Learn what that is and why trie is not a typo in this resource article.
---
This article will discuss how the data structures that organize claims by names work, how proofs are generated/verified, and how consensus on the state of the trie is represented.
For looking into how claims end up in the trie, [read this instead](/spec#claimtrie).
## The Trie
A Trie is an ordered tree data structure. Think of it as a hash map or dictionary where the keys are ordered and organized in prefixes. It's used for storing claims by name and looks (a bit) like this:
![Wikipedia claimtrie](https://upload.wikimedia.org/wikipedia/commons/thumb/b/be/Trie_example.svg/400px-Trie_example.svg.png)
You can read more on it [here](https://en.wikipedia.org/wiki/Trie), but for understanding the ClaimTrie let's just think of it as a mapping of names and claims where you can easily find which character of a name prefix forms a collision. Like the first `t` for `{to, tea, ted, ten}` and the `e` in `te` for `{tea, ted, ten}`.
## Consensus
Each block header holds an extra 256 bits value calculated out of the root node of the claim trie at that block height. It's called `nameclaimroot` and is influenced by all children nodes as we will see next. If a blockchain network peer disagrees that a claim name was accepted or who is the winner of each name, its `nameclaimroot` will differ and the block won't form the same chain as the ones that accepted the official rules. This is the same for the traditional Merkle root, which is the root of the [Merkle tree](https://bitcoin.org/en/glossary/merkle-tree), formed by transactions in a block.
## What's in a leaf?
The leaf currently holds the winner of that name. It's formed by the transaction hash, output number of the claim in that transaction and the height it was accepted.
### Generating the leaf hash
So, let's suppose that the winner claim of `mindblown` name was made at transaction output `1` of the transaction hash `67ad533eb2676c9d36bfa100092af5358de747e08ef928c0c54a8b3891c2b76b` and included in the Trie at height `102`.
1. The transaction hash is converted from [RPC byte order](https://bitcoin.org/en/glossary/rpc-byte-order) to [internal byte order](https://bitcoin.org/en/glossary/internal-byte-order).
2. The output number becomes a simple string.
3. The height becomes a big endian 64 bits value.
4. The node hash is calculated as the double SHA-256 hash of the double SHA-256 hash of the internal byte order representation of the transaction hash concatenated with the double SHA-256 hash of the output number representation concatenated with the double SHA-256 hash of the height.
This is better represented in the simple python script below:
```python
import struct
from hashlib import sha256
from binascii import unhexlify, hexlify
name = "mindblown".encode()
tx = '67ad533eb2676c9d36bfa100092af5358de747e08ef928c0c54a8b3891c2b76b'
nout = 1
at_height = 102
sha256d = lambda x: sha256(sha256(x).digest()).digest()
def hash_leaf(tx, nout, height):
raw_tx = unhexlify(tx)[::-1]
raw_nout = str(nout).encode()
raw_height = struct.pack('>Q', height)
return sha256d(sha256d(raw_tx) + sha256d(raw_nout) + sha256d(raw_height))
print("leaf hash is {}".format(hexlify(hash_leaf(tx, nout, at_height)[::-1])))
```
## How is the root hash calculated?
Let's start with a ClaimTrie holding a single claim.
The claim is named `mindblown` and was included at block 102 with the same details as the last section. It's actually a real claim as [you can see in the block explorer](https://explorer.lbry.com/blocks/102). This block has the first claim ever, so the ClaimTrie was the simple case being explained in here.
We start with the leaf hash:
1. Hash the leaf hash using double SHA-256.
2. For each character of the name (nodes of the trie), the hash of a node is the double SHA-256 of the node's character concatenated with the children hash.
Continuing with the Python script from the last section, this is how to calculate the root of a claim trie holding a single name:
```python
def hash_single_claim_trie(name, claim_tx, claim_nout, claim_height):
final_hash = sha256d(hash_leaf(claim_tx, claim_nout, claim_height))
for character in reversed(name):
final_hash = sha256d(bytearray([character]) + final_hash)
return final_hash
print("root hash is {}".format(hexlify(hash_single_claim_trie(name, tx, nout, at_height)[::-1])))
```
## What if there are more leafs?
Just concatenate the node character with the children’s hashes sha256d(character + leftmost child hash + ... + rightmost child hash ).
TO BE IMPROVED

21
src/content/resources/consensus.md Normal file
View file

@ -0,0 +1,21 @@
---
title: Consensus Algorithm
description: How does the LBRY blockchain achieve consensus? This resource page will explain.
---
LBRY uses [proof of work](https://en.bitcoin.it/wiki/Proof_of_work) as a [consensus mechanism](/spec#consensus), the same way that Bitcoin does.
LBRY has differences in hash function, block targeting, and difficulty adjustment.
### Hash Mechanism
```python
intermediate = sha512(sha256(sha256(data))) # compute the sha512() of the double-sha256() of the data
left = ripemd(intermediate[:len(intermediate)/2]) # separately ripemd160 the left half
right = ripemd(intermediate[len(intermediate)/2:]) # and the right half
proof = sha256(sha256(left + right)) # concatenate the two halves, and double-sha256() it again
```
### Block Targeting & Difficulty Adjustment
The targeted time of each LBRY block is 2.5 mintues (150 seconds). More information and links to source code [here](https://lbry.tech/spec#consensus).

75
src/content/resources/daemon-settings.mdx Normal file
View file

@ -0,0 +1,75 @@
---
title: SDK Settings
description: The daemon provided by the LBRY SDK has many settings. This resource lists them all and what they mean. Ready, set, settings!
---
This document outlines how to configure SDK daemon settings and what options are available. They can be found on the lbry GitHub repository in [conf.py](https://github.com/lbryio/lbry-sdk/blob/master/lbry/conf.py).
## Daemon settings configuration
The easiest way to configure the settings is by editing the `daemon_settings.yml` file (may need to be created) that resides in the default [lbrynet directory](https://lbry.com/faq/lbry-directories). These settings can also be configured via the [settings_set](https://lbry.tech/api/sdk#settings_set) API call. The [settings_get](https://lbry.tech/api/sdk#settings_get) API call can be used to retrieve current values. Some values will require an SDK restart after being set via the API call.
Sample daemon_settings.yml file:
```
tcp_port: 3335
lbryum_servers: ['spv11.lbry.com:50001','spv19.lbry.com:50001']
download_directory: 'c:\lbry\Downloads'
use_upnp: false
```
To run the SDK with a specific configuration file, launch it by passing the config path: ```lbrynet start --config=c:\path\to\conf\daemon_settings.yml```. To run in debug mode, start with ```lbrynet start --verbose=lbrynet```.
## Configuration options
Configuration options are organized by their respective areas: Files, Wallet, Network, Security and Other.
### Files
| Setting | Format | Default value | Sample Values | Description |
|------------------------|---------|-------------------------------------------------------|--------------------|--------------------------------------------------------------------------------------|
| data_dir | string | [varies by OS](https://lbry.com/faq/lbry-directories) | 'c:\lbry\lbrynet\' | Where to store the lbrynet folder, which includes blob files, logs and config data |
| delete_blobs_on_remove | boolean | true | false | Delete blobs on a file_delete call? |
| download_dir | string | local downloads folder | 'c:\lbry\lbrynet\' | Location of downloaded output files |
### Wallet
| Setting | Format | Default value | Sample Values | Description |
|-----------------|--------|-------------------------------------------------------|----------------------------|------------------------------------------------------------|
| blockchain_name | string | 'lbrycrd_main' | 'lbrycrd_regtest' | Blockchain network to connect to |
| lbryum_servers | list | [‘spv11.lbry.com:50001’,‘spv19.lbry.com:50001’] | ["mylbryum.lbry.com:50001] | SPV wallet server address(Default servers are spv11-spv19) |
| wallet_dir | string | [varies by OS](https://lbry.com/faq/lbry-directories) | 'c:\lbry\lbryum\' | Wallet data location |
| max_key_fee | json | \{'currency': 'USD', 'amount': 50.0} | \{'currency': 'LBC', 'amount': 5.0} | Max payment allowed for content |
| wallet | string | 'lbryum' | 'lbrycrd' | Choice of wallet software, SPV (lbryum) vs full node (lbrycrd). Currently only lbryum supported |
| use_keyring | boolean | false | true | Store wallet password in keyring (not currently available) |
### Network
| Setting | Format | Default value | Sample Values | Description |
|------------------------------|---------|----------------------------|--------------------------|----------------------------------------------------------------------------------|
| api | string | localhost:5279 | 0:0:0:0:5280 | IP address and port the SDK API will listen on |
| streaming_server | string | localhost:5280 | 0:0:0:0:5280 | IP address and port the media/streaming server will listen on |
| cache_time | integer | 150 | 90 | How long to keep resolve data in cache |
| data_rate | float | 0.0001 | 0.05 | What LBC rate, per MB, to offer DHT data at (currently disabled in the protocol) |
| udp_port | integer | 4444 | 4445 | UDP port used to announce blobs |
| download_timeout | integer | 30 | 60 | Time, in seconds, to allow get call to resolve and get initial blobs |
| blob_download_timeout | integer | 30 | 60 | Time, in seconds, to allow download to get next blob |
| announce_head_blobs_only | boolean | true | false | Only announce first data blob |
| concurrent_blob_announcers | integer | 10 | 0 | Threads used in order to announce blobs. 0 means disabled |
| known_dht_nodes | list | ['lbrynet1.lbry.com:4444'] | ['myDHT.lbry.com:4444'] | Bootstrap nodes for network connectivity |
| max_connections_per_download | integer | 5 | 10 | Threads used to download blobs |
| seek_head_blob_first | boolean | true | false | Search for first data blob after downloading sd blob |
| tcp_port | integer | 4444 | 3334 | Port the SDK will listen on |
| concurrent_reflector_uploads | integer | 5 | 10 | Connections to use while uploading data to reflector |
| reflect_streams | boolean | true | false | Send published data to reflector servers |
| reflector_servers | list | ['reflector.lbry.com'] | ['myreflector.lbry.com'] | Server data will be reflected to |
| fixed-peer-delay | integer | 2 | 5 | Time, in mintues, to allow download from P2P before trying fixed peer |
| peer_connect_timeout | integer | 30 | 15 | Time, in seconds, to allow download to find peers |
| node_rpc_timeout | integer | 5 | 10 | Time, in seconds, to allow connection over DHT |
| network_interface | string | 0:0:0:0 | 127.0.0.1 | Interface to use for the DHT and blob exchange |
| use_upnp | boolean | true | false | Attempt external port mapping via UPnP |
| streaming_get | boolean | false | true | Allow calling localhost:5280/get/claimname requests |
| save_files | boolean | true | false | Save files with each download |
| save_blobs | boolean | true | false | Save blobs with each download |
### Other
| Setting | Format | Default value | Sample Values | Description |
|--------------------|---------|---------------|--------------------------------|------------------------------------------------------------------------------------------------------------------|
| components_to_skip | list | [] | ['reflector','hash_announcer'] | Disable components, [see entire list here](https://github.com/lbryio/lbry-sdk/wiki/Component-Dependencies-Table) |
| share_usage_data | boolean | true | false | Share analytics data |

39
src/content/resources/dht-bootstrap.md Normal file
View file

@ -0,0 +1,39 @@
---
title: Hosting a DHT bootstrap node
description: How to setup a bootstrap DHT node
---
This guide will help you setup and maintain a LBRY DHT [bootstrap node](https://en.wikipedia.org/wiki/Bootstrapping_node). Those nodes are important so people can join the network on first startup.
After finishing and checking that it works, if you want to add your node to the SDK bootstrap list just open a PR adding yourself to the [conf file](https://github.com/lbryio/lbry-sdk/blob/master/lbry/conf.py#L694) or an issue on the [SDK repo](https://github.com/lbryio/lbry-sdk/).
## Requirements
- Being reachable over UDP on the internet at some port
- 1GB of memory
- Docker or Python 3.7 (check [pyenv](https://github.com/pyenv/pyenv) if your Linux distribution doesn't offer that version)
## Running directly from Docker
This is the easiest way to run and maintain your node. Just run:
```bash
docker run -d -p 4444:4444 lbry/dht-bootstrap:latest
```
## Installing LBRY SDK from source
The most up to date guide for doing it will always be in the [INSTALL.md file](https://github.com/lbryio/lbry-sdk/blob/master/INSTALL.md). Please refer to it if you run into trouble. Otherwise, this should be enough most of the time (assuming requirements are all there):
```bash
git clone https://github.com/lbryio/lbry-sdk.git
make install
```
### Running a node from source
After installing, just:
```bash
python scripts/dht_node.py
```
### Checking if it is working
From another machine with the SDK installed, run:
```bash
python scripts/dht_node.py --bootstrap_node your-server-domain-here.com:4444
```
After 10-20 seconds, you should see more than 0 peers on the log messages. If that is not the case, check firewall on the bootstrap node and see if it is reachable.

93
src/content/resources/download-overview-new.md Normal file
View file

@ -0,0 +1,93 @@
---
title: Content Downloading
description: This resource article walks through the step-by-step process of downloading a piece of content from the LBRY network.
---
This resource outlines the step-by-step process of using the LBRY protocol to download something.
## Getting Started
Downloading spans the three core components of LBRY (blockchain, DHT, blob exchange) and explains the structure of the claim metadata and the blobs that make up a stream.
You will need:
- a running [lbrycrd](https://github.com/lbryio/lbrycrd) node, or another way to access blockchain data (perhaps [lbryumx](https://github.com/lbryio/lbryumx) or [chainquery](https://github.com/lbryio/chainquery))
- a running DHT node
- the claim ID of the content you wish to download
For this example, we will use claim ID `d9317ac7842f88ba442fee749c4f834353c24206`.
## Overview
- start with **claim ID**
- blockchain gets you **metadata** for your claim ID
- parsing metadata gets you **stream hash** and fee info
- if there is a fee, pay it using the blockchain
- dht gets you **peers** for the stream hash
- blob exchange gets **manifest blob** from peers
- manifest is parsed to get **content blob hashes**
- dht and then blob exchange get you the **content blobs**
- blobs are decrypted and assembled to create the **file**
## Parse the Metadata
Perform a `getclaimbyid` call to lbrycrd using the claim ID for the claim you want to look up. You should get a response with some parameters. The `value` parameter contains the claim contents as a protobuf-encoded binary string. Decode the value using the protobuf definitions in [lbryio/types](https://github.com/lbryio/types/tree/master/v2/proto). You will get a Claim object.
## Get the Stream Hash
Confirm that `Claim.type` is `1` (a stream claim).
Claim.Stream.hash contains the stream hash, which is the hash of the manifest blob in the stream.
## Pay the Fee
Check the `Claim.Stream.Fee` field. If it exists, then a payment is required to download this content. Get the address, amount, and currency from the Fee, convert the amount to LBC if its not already in LBC, and perform a `sendtoaddress` call to lbrycrd to send the fee amount to the fee address.
Our example claim does not have a fee. If you want to see a claim with a fee, look up claim ID `fbdcd44a97810522d23d5f1335b8ca04be9d776c`.
## Find Hosts for the Manifest Blob
Look up the stream hash in the DHT. Internally this will perform an iterativeFindValue call, starting with the nodes already in the routing table and proceeding to nodes that are closest to the stream hash. The DHT should return a list of hosts that have advertised that they have this hash.
## Download Manifest Blob
Use the blob exchange protocol to request the manifest blob from the hosts found in the previous step.
## Read Manifest Blob
The manifest is JSON-formatted text. It contains a dictionary with the following structure:
```
{
"blobs": [
{
"blobHash": "b7e43c102781f978c24bc2bc...",
"iv": "63a6befc3c8d01f662ffad2f2381b357",
"length": 2097152,
},
...
],
"filename": "574c707655476a766d58632e6d7034",
"key": "ee768c4e642012bb5b2e20cf9b1f997b",
"version":1
}
```
All string fields are hex-encoded.
To download the content blobs, repeat the steps we took for the stream hash, but instead use the `blobHash` value for each blob. Look up the blob hash in the DHT to find hosts, then download the blob from those hosts.
## Decrypt and Assemble Blobs
Now that all the blobs have been downloaded, they can be decrypted and assembled into the original file. Decrypt each blob using the key and IVs in the manifest, and concatenate the decrypted bytes in order. Write the finished file to disk.
## Enjoy a Cute Puppy Video
You earned it.

99
src/content/resources/download-overview.mdx Normal file
View file

@ -0,0 +1,99 @@
---
title: Content Downloading
description: This resource article walks through the step-by-step process of downloading a piece of content from the LBRY network.
---
This resource outlines the step-by-step process of using the LBRY protocol to download something.
## Getting Started
Downloading spans the three core components of LBRY (blockchain, DHT, blob exchange) and explains the structure of the [claim metadata](/spec#metadata) and the blobs that make up a [LBRY stream](/spec#data).
You will need:
- a running [lbrycrd](https://github.com/lbryio/lbrycrd) node, or another way to access blockchain data (perhaps [lbryumx](https://github.com/lbryio/lbryumx) or [chainquery](https://github.com/lbryio/chainquery))
- a running DHT node
- the `claimID` of the content you wish to download
For this example, we will use claimID `d9317ac7842f88ba442fee749c4f834353c24206`.
## Overview
- start with **claim ID**
- blockchain gets you **metadata** for your claimID
- parsing metadata gets you **sd hash** and fee info
- if there is a fee, pay it using the blockchain
- dht gets you **peers** for the sd hash
- blob exchange gets **sd blob** from peers
- sd blob is parsed to get **content hashes**
- dht and then blob exchange get you the **content blobs**
- blobs are decrypted and assembled to create the **file**
## Parse the Metadata
Perform a `getclaimbyid` call to lbrycrd using the claimID for the claim you want to look up. You should get a response with some parameters. The `value` parameter contains the claim contents as a protobuf-encoded binary string. Decode the value using the protobuf definitions in [lbryio/types](https://github.com/lbryio/types/tree/master/v2/proto). You will get a Claim object.
## Get the SD Hash
Confirm that `Claim.claimType` is `1` (streamType) and `Claim.Stream.Source.sourceType` is `1` (lbry_sd_hash).
Claim.Stream.Source.source contains the hash of the `sd blob` for the content. We call this the `sd hash`.
## Pay the Fee
Check the `Claim.Stream.Metadata.Fee` field. If it exists, then a payment is required to download this content. Get the address, amount, and currency from the Fee, convert the amount to LBC if its not already in LBC, and perform a `sendtoaddress` call to lbrycrd to send the fee amount to the fee address.
Our example claim does not have a fee. If you want to see a claim with a fee, look up claim ID `fbdcd44a97810522d23d5f1335b8ca04be9d776c`.
## Find Hosts for SD Blob
Look up the sd hash in the DHT. Internally this will perform an iterativeFindValue call, starting with the nodes already in the routing table and proceeding to nodes that are closest to the sd hash. The DHT should return a list of hosts that have advertised that they have this hash.
## Download SD Blob
Use blob exchange protocol to request the sd blob from the hosts found in the previous step.
## Read SD Blob
The SD blob is JSON-formatted text. It contains a dictionary with the following structure:
```json
{
"stream_name": "574c707655476a766d58632e6d7034",
"blobs": [
{
"length": 2097152,
"blob_num": 6,
"blob_hash": "b7e43c102781f978c24bc2bc...",
"iv": "63a6befc3c8d01f662ffad2f2381b357"
},
...
],
"stream_type": "lbryfile",
"key": "ee768c4e642012bb5b2e20cf9b1f997b",
"suggested_file_name": "574c707655476a766d58632e6d7034",
"stream_hash": "6b1f9d5f129e06bb94b2ffdda817a5848c...",
}
```
All string fields are hex-encoded.
To download the content blobs, repeat the steps we took for the SD hash, but instead use the `blob_hash` value for each blob. Look up the blob_hash in the DHT to find hosts, then download the blob from those hosts.
Every stream has 0-length blob as the last blob in the list of blobs. This blob is not a real blob, and is not available on the network. It serves a similar purpose to the null byte at the end of a string - it signals that this is the last blob. This supports streaming a file when the number of blobs is not known in advance.
## Decrypt and Assemble Blobs
Now that all the blobs have been downloaded, they can be decrypted and assembled into the original file. Decrypt each blob using the key and IVs in the SD blob, and concatenate the decrypted bytes in `blob_num` order. Write the finished file to disk (you may use the `suggested_file_name` from the SD blob, or choose your own).
## Enjoy a Cute Puppy Video
You earned it.

14
src/content/resources/encrypt-lbrycrd.md Normal file
View file

@ -0,0 +1,14 @@
---
title: Full Node Encryption
description: Learn how to use encryption with lbrycrd, the full blockchain software for the LBRY network.
---
*Note: The below instructions are the [LBRYCrd Full Blockchain wallet](https://github.com/lbryio/lbrycrd) and not the default wallet that ships with the LBRY App. We are still working on an encryption solution for this.*
You can use `lbrycrd-cli encryptwallet <passphrase>` to encrypt your wallet.
You can use `lbrycrd-cli walletpassphrase <passphrase> <timeout>` to temporarily unlock the wallet. The <timeout> parameter is in seconds.
For example, `lbrycrd-cli walletpassphrase 'open sesame 321' 300` would unlock your wallet for five minutes, assuming your passphrase was `open sesame 321`. (In reality, you should choose a harder-to-guess passphrase than that.)
If you set <timeout> too low, it might expire before you get done using your wallet. If you set it too high, you might forget that you left your wallet unlocked.

65
src/content/resources/p2p-seeding.md Normal file
View file

@ -0,0 +1,65 @@
---
title: "LBRY P2P: Settings and troubleshooting"
description: Guide on properly setting up P2P nodes and how to diagnose/fix common issues.
---
# LBRY P2P: Settings and troubleshooting
A very important step in supporting the network resilience is hosting content in a decentralized way. This already happens when you stream content using the desktop version of LBRY APP, but sometimes we want to make sure this is working or dedicate larger amounts of resources for this task.
This document aims to explain P2P configuration and troubleshooting from small to large nodes. If you don't know how to change SDK settings, check [this other document first](https://lbry.tech/resources/daemon-settings).
## Reachability
The first priority when seeding content is making sure there is a way for other nodes to reach you across the internet.
### Figuring out your ports
In order to troubleshoot reachability, we start by checking your configuration for the UDP and TCP ports. By default, they will both be set to 4444. Those can be found on the configuration under the keys `udp_port` and `tcp_port`. Please set them both to the same value as this helps connectivity trough [hole punching](https://en.wikipedia.org/wiki/Hole_punching_(networking)) and ease of management.
### Checking for reachability
There are some websites providing generic ways to check ports, like:
- https://www.portcheckers.com/
- https://portchecker.co/check
However, checking the port does not check if LBRY P2P protocol is working behind it. For a better check, we can use a tool hosted by Madiator, a community member.
- To test for UDP (DHT): http://test.madiator.com:60666/dht/<your `udp_port`>
- To test for TCP (P2P): http://test.madiator.com:60666/p2p/<your `tcp_port`>
As a last resource to test a remote machine DHT service, from a local SDK try:
```bash
lbrynet peer ping <DHT node id> <IP> <port>
```
To find out what the `DHT node id` is, on the target machine run `lbrynet status` and look for `node_id` .
### I am unreachable. What now?
VPN users: check with your provider if they feature port forwarding. There are guides specific to each one, like [this one from Mullvad](https://mullvad.net/en/help/port-forwarding-and-mullvad/).
Domestic routers: there are websites like [this one](https://portforward.com/how-to-port-forward/) providing information on popular router models. Unfortunately, this document would be huge if we added port forwarding instructions for every router/firewall.
Servers: check if your firewall is blocking the SDK ports. For ufw on Linux, this is `sudo ufw allow <port>`.
If you still have trouble figuring that out, don't be shy, [ask the LBRY community on Discord!](https://chat.lbry.com/)
## Content blobs storage settings
Files in LBRY are composed by `content blobs`, which can be seen as chunks of binary encrypted data belonging to some content. By default, the SDK enables saving blobs to disk, which then can be served over P2P. To check if that is enabled, look for the `save_blobs` setting.
**The following settings are isolated. The space limit set for one does not apply to the other.**
### Setting up storage space control
By default, content blobs are kept as long as the files are still in your file list. If you wish to allocate a space limit for content blobs and let the SDK decide what to delete, set `blob_storage_limit` to a value in megabytes.
This won't delete your downloads from the file list. Instead, it deletes content blobs associated with older files as space for newer blobs is requested.
### Setting up space for automatic contribution
This section is aimed at fully automatic contribution in background. Normal usage of the SDK with P2P enabled already helps the network, but requires interaction.
LBRY SDK can be configured to help the P2P network by automatically downloading and hosting content. This is ideal for contributing spare disk space and network bandwidth without further interaction.
Enabling: change `network_storage_limit` to the size (in megabytes) that will be used for automatic seeding.
Disabling: setting the space to 0 disables it. The space used will eventually be released automatically. For cleaning immediately, issue a `lbrynet blob clean` from command line.

104
src/content/resources/repository-standards.md Normal file
View file

@ -0,0 +1,104 @@
---
title: Repository Documentation Standards
description: All repository documentation at LBRY complies with a single standard, outlined in this resource article.
---
This document outlines the standards for all public, open-source repositories used by LBRY.
The goal of this document is to ensure that LBRY documentation is welcoming, informative, comprehensive, and well-written.
Each codebase should include the following documents committed as markdown files at the top level of the repository:
## README.md
This document exists to introduce a project to a new visitor. It may also serve as a reference document for existing contributors. This document should include the following:
### Title / Heading
* The title (name) of the project at the top as an h1
* Any build status badges as appropriate immediately below the title
* A one or two sentence description of the project, below the title or status badges. The description should mention the language and any key frameworks (e.g. "lbry" should mention it uses both Python and Twisted)
* A screenshot, image, or gif of the project below the description (host it on spee.ch!)
* A table of contents if the document is long (can be one line of links to sections)
### Installation
* A single header labeled "Install" should be the next header as an h2
* Installation means installation for users. It should cover how to have the software on their computer in a user-friendly fashion, not how to run from source (e.g. it should link binaries if they exist)
* If the project is a library only intended to be used in other projects, it should still exist but can be extremely succinct (e.g. pip install lbryschema)
* If the project is not designed to be installed directly or used as a library in other projects, it should state as such (e.g. "This project is not designed to be installed directly. Continue reading below to learn how to use this project.”)
### Usage
* A single header labeled "Usage" should be the next header as an h2
* The Usage section should explain how to run or launch the project if applicable
* The Usage section should include examples of some basic commands if applicable
* In the case of a library, it should provide a few basic examples of how the library would be used
* Usage should always be included, even if it’s very simple (e.g. "Double click the installed application to browse with the LBRY network.”)
### Running from Source
* This section covers how to run the project from source and/or with the intent of working directly on it
* It can have a Prerequisites section for what is required to run from source
* It is okay to assume some basic assumptions about what people know about the language or tools the project uses. However, it’s good whenever possible to assume very little and provide links to the user for how to get the prerequisites
* For prerequisites for which it is not safe to assume knowledge of, the project should list the explicit command to add the prerequisite, or link to instructions on how to add the prerequisite
* If there are operating system specific instructions, these should be broken out into separate headings for each operating system
* It is okay to assume Linux and Unix are "first class citizens". While we want to support all OSs, Windows instructions and considerations can be secondary
* Include a step that explains how to verify you are running from source directly
* It is okay to point to a INSTALL.md file (which should reside in the root folder) if the installation steps are relatively lengthy
### Contributing
* A single header labeled "Contributing" should appear as an h2
* This should be the same message: "Contributions to this project are welcome, encouraged, and compensated. For more details, see *[CONTRIBUTING.md](CONTRIBUTING.md)*.”
* If CONTRIBUTING.md does not exist in the project, it should link to [https://lbry.com/faq/contributing](https://lbry.com/faq/contributing) (soon to be lbry.tech/contributing)
### (Additional Headings)
* Additional headings as appropriate will typically exist after the above and below the subsequent areas
* These areas should cover anything else appropriate or helpful to introduce a project to a new user
### License
* A single header labeled "License" should appear as an h2
* Assuming a standard license, this should be the same two sentences: "This project is X licensed. For the full license, see [LICENSE]."
### Security
* "We take security seriously. Please contact [security@lbry.com](mailto:security@lbry.com) regarding any security issues. Our PGP key is [here](https://lbry.com/faq/pgp-key) if you need it."
### Contact
* A single header labeled "Contact" should appear as an h2
* This should be the same or a similar message to the following: "The primary contact for this project is [@XXX](https://github.com/@XXX) ([xxx@lbry.com](mailto:xxx@lbry.com))"
### Additional Info and Links
* (optional)
* References to any additional documentation, such as generated API docs
## CONTRIBUTING.md
This document explains anything a visitor would need to know to contribute to the project.
This document should cover the following:
* First, it should contain a single sentence: "This project follows the global contributing standards for all LBRY projects, to read those go < here >.”
* A "Code Overview" section explaining some basic design choices and how to begin stepping through the code. An example would be explaining that Daemon.py is the primary entry point for the daemon code, and one can begin to trace through the code by looking for jsonrpc_xxx, where xxx is one of the api calls listed [here](https://lbry.com/api)
* A "Testing" section explaining how to run tests and stating that tests are necessary
* Information on how to submit pull requests, and what to expect afterwards (e.g. a link to our [branching doc](https://github.com/lbryio/lbry-sdk/wiki/Branching-and-Merging), commands to run before submitting PR, tests must pass, changelog entry, etc). If you find this gets repetitive, it may be best to link to a global doc
* Anything else a new visitor to a repository should know about contributing to that specific repository (linting, generating documentation, etc)
## LICENSE
Every repository should have a LICENSE file stating the license. The default license we use is MIT, and we license all code as MIT whenever possible.
Some code may use external libraries that prevent MIT licensing. If adding a license to a project for the first time that uses 3rd-party code, please ensure it is okay to actually MIT license it.
## ISSUE_TEMPLATE.md
A template for issues should exist to guide users in correctly filing them.
## Style and Formatting Notes
- Rely on autowrap instead of manually breaking up paragraphs with a carriage return.

20
src/content/resources/setup-videos.md Normal file
View file

@ -0,0 +1,20 @@
---
title: Developer Setup Intro
---
# Getting started with LBRY development
Having trouble with getting your LBRY development environment setup or do you prefer following along with a video? Well you're in luck...we have setup videos for many of projects!
## Desktop Application
Check out [our video tutorial](/resources/video-lbrydesktop) to setup the [Desktop app](https://github.com/lbryio/lbry-desktop) development environment.
## Android Application
Check out [our video tutorial](/resources/video-lbryandroid) to setup the [Android app](https://github.com/lbryio/lbry-android) development environment.
## LBRY SDK
Check out [our video tutorial](/resources/video-lbrysdk) to setup the [LBRY SDK](https://github.com/lbryio/lbry-sdk) development environment.
## LBRY Blockchain
Check out [our video tutorial](/resources/video-lbrycrd) to setup the [LBRY Blockchain](https://github.com/lbryio/lbrycrd) development environment.

61
src/content/resources/time-locked-transactions.md Normal file
View file

@ -0,0 +1,61 @@
---
title: How to spend your time locked transaction
---
This guide will walk you through the process of claiming a time locked transaction from a USB key. This involves accessing the transaction details on the key, making sure you have the latest version of `lbrynet` and finally using the transaction details to call `account_deposit` to claim your LBC.
## Check `lbrynet` version
If you already have `lbrynet` installed then you can check your version like this:
```
lbrynet version
```
If above command fails, you may need to start `lbrynet` first (and then try above again):
```
lbrynet start
```
If you do not have `lbrynet` installed or your version is less than `v0.108.0` then you can get latest version here:
[https://github.com/lbryio/lbry-sdk/releases](https://github.com/lbryio/lbry-sdk/releases)
## Gather Information
### Transaction ID and Transaction Output Number
1. On the USB key, find a file named `address.txt` and copy the address in this file.
1. Go to [LBRY Explorer](https://explorer.lbry.com/) and enter the address you copied.
1. You should see one transaction containing this address, click on this transaction.
1. You will need two pieces of information on this page, the `transaction id` and the `nout`.
1. The `transaction id` can be found at the top of the page and directly below the text `LBRY Transaction`.
1. The `nout` is the position of the output containing your address, starting with 0. Starting from the top of the list of outputs, count the outputs until you get to your address, then subtract 1 from this count, that is your `nout`.
### Private Key and Redeem Script
1. On the USB key, find a file named `key.zip` and unzip this file using the password emailed to you previously.
1. You should now have a file named `key.txt` which is base64 encoded and contains your `private key` and `redeem script`.
1. To decode the contents of the file you can use a website such as [base64decode.org](https://www.base64decode.org/) (not a secure option) or if you have Python installed you can do this on the command line:
```
python -m base64 -d /path/to/key.txt
```
1. After decoding you will see a key/value mapping of various items, including `privateKey` and `redeemScript`. Take note of these values.
## Redeem
Now that you have gathered the necessary information it is easy to redeem your LBC. Time locked transaction can be redeemed using the `lbrynet account deposit` command (fill in the values you gathered previously):
```
lbrynet account deposit <transaction id> <nout> <redeemScript> <privateKey>
```
If you get an error that says `AssertionError: Cannot find private key for signing output.`, try a different number for `<nout>` (for example, increase or decrease it by 1).
Enjoy your LBC!
## Get in touch
Whether you got to the end without a hiccup or you got stuck along the way, we want to hear from you. [Join our Discord](https://discord.gg/y3W9JuS) to get help, stay updated, and talk to other wallet server operators.

214
src/content/resources/wallet-server.md Normal file
View file

@ -0,0 +1,214 @@
---
title: How To Run Your Own Wallet Server
---
This guide will walk you through the process of setting up a LBRY wallet server. This involves provisioning a web server and setting up some services (docker, lbrycrd, and the wallet server). At the end, you'll have your own connection to the LBRY network.
**note:** This is early-stage stuff. You may encounter unexpected issues. Please be patient and don't hesitate to [reach out for help](#get-in-touch).
## Start With A Fresh Server
We recommend a quad-core server with at least 16GB RAM, 200GB disk, and a fresh Ubuntu 18.04 install. Memory usage is flexible. 32 GB works best, but 16 GB is enough for a few clients.
Make sure your firewall has ports 9246 and 50001 open. 9246 is the port lbrycrd uses to communicate to other nodes. 50001 is the wallet server RPC port.
## Install lbrycrd
### Download and setup
Download the [latest release of lbrycrd](https://github.com/lbryio/lbrycrd/releases/latest).
Then, create a folder on your home directory called `.lbrycrd` and save the following to `.lbrycrd/lbrycrd.conf`:
```
txindex=1
server=1
daemon=1
rpcuser=lbry
rpcpassword=lbry
dustrelayfee=0.00000001
```
Feel free to change the `rpcuser` or `rpcpassword`. If you do, you'll have to update the `DAEMON_URL` variable later on (in the docker-compose.yml file) to match the user/password you chose.
## Create a service (optional)
You can run lbrycrdd directly using `./lbrycrdd`. However, we recommend creatinga systemd service to manage the process for you.
Create a file at `/etc/systemd/system/lbrycrdd.service` with the following contents:
```
[Unit]
Description="LBRYcrd daemon"
After=network.target
[Service]
ExecStart=/home/<your_user>/lbrycrdd -datadir="/home/<your_user>/.lbrycrd" -pid="/run/lbrycrdd/lbrycrdd.pid"
# Creates /run/lbrycrdd
RuntimeDirectory=lbrycrdd
Type=Forking
PIDFile=/run/lbrycrdd/lbrycrdd.pid
Restart=on-failure
# hardening
PrivateTmp=true
ProtectSystem=full
NoNewPrivileges=true
PrivateDevices=true
MemoryDenyWriteExecute=true
[Install]
WantedBy=multi-user.target
```
Then run `sudo systemctl daemon-reload`.
Now you can start and stop lbrycrd with `sudo service lbrycrdd start` and `sudo service lbrycrdd stop`.
You can watch the lbrycrd log with `tail -f ~/.lbrycrd/debug.log`
## Set Up Docker
### Install Docker & Docker Compose
```
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common && \
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - && \
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" && \
sudo apt install -y docker-ce docker-ce-cli containerd.io && \
sudo systemctl enable docker && sudo systemctl start docker && \
sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && \
sudo chmod +x /usr/local/bin/docker-compose
sudo usermod -aG docker $USER
```
### Download our example docker-compose.yml
You can see it [here](https://github.com/lbryio/lbry-sdk/blob/master/docker/docker-compose-wallet-server.yml).
```
curl -L "https://raw.githubusercontent.com/lbryio/lbry-sdk/master/docker/docker-compose-wallet-server.yml" -o docker-compose.yml
```
Make sure the user and password in the `DAEMON_URL` variable (the `lbry@lbry` part) in this docker-compose.yml matches the user/password in your `~/.lbrycrd/lbrycrd.conf` file.
### Download snapshots for elasticsearch and the wallet server (optional)
You can skip the initial sync by starting from a snapshot. The following will download a snapshot of the elasticsearch volume and move it into the default location for docker volumes on ubuntu, on other systems you may need to adjust the path used here. Note: snapshot heights must be the same. The tars can be deleted after setting the volumes up.
```bash
SNAPSHOT_HEIGHT="1049658"
ES_VOLUME_PATH="/var/lib/docker/volumes/${USER}_es01"
ES_SNAPSHOT_TAR_NAME="es_snapshot_${SNAPSHOT_HEIGHT}.tar"
ES_SNAPSHOT_URL="https://snapshots.lbry.com/hub/${ES_SNAPSHOT_TAR_NAME}"
wget $ES_SNAPSHOT_URL
echo "decompressing elasticsearch snapshot"
tar -xf $ES_SNAPSHOT_TAR_NAME
sudo chown -R $USER:root "snapshot_es_${SNAPSHOT_HEIGHT}"
sudo chmod -R 775 "snapshot_es_${SNAPSHOT_HEIGHT}"
sudo mkdir -p $ES_VOLUME_PATH
sudo rm -rf "${ES_VOLUME_PATH}/_data"
sudo mv "snapshot_es_${SNAPSHOT_HEIGHT}" "${ES_VOLUME_PATH}/_data"
```
The following will download the wallet server docker volume and move it into place as well.
```bash
echo "fetching wallet server snapshot"
SNAPSHOT_HEIGHT="1049658"
HUB_VOLUME_PATH="/var/lib/docker/volumes/${USER}_wallet_server"
SNAPSHOT_TAR_NAME="wallet_server_snapshot_${SNAPSHOT_HEIGHT}.tar"
SNAPSHOT_URL="https://snapshots.lbry.com/hub/${SNAPSHOT_TAR_NAME}"
wget $SNAPSHOT_URL
tar -xf $SNAPSHOT_TAR_NAME
sudo mkdir -p $HUB_VOLUME_PATH
sudo rm -rf "${HUB_VOLUME_PATH}/_data"
sudo chown -R 999:999 "snapshot_${SNAPSHOT_HEIGHT}"
sudo mv "snapshot_${SNAPSHOT_HEIGHT}" "${HUB_VOLUME_PATH}/_data"
```
## Turn It On
### Start the servers
```
docker-compose up --detach
```
### Check that everything worked
The first time you start the wallet server, it will take a few minutes to download a recent snapshot of the database and extract it. You can follow the progress with
```
docker-compose logs --follow
```
After the wallet server has caught up, it will bind to port 50001 and start responding to requests. You can check if this happened by running
```
sudo netstat -tlpn | grep 50001
```
If there is no output, the port is ont bound yet and the server is still catching up. Check the logs for more info.
After the wallet server is ready, check that it responds to basic RPC calls:
```
echo '{"id":1,"method":"server.version"}' | timeout 1 curl telnet://localhost:50001
```
You should see a response like `{"jsonrpc": "2.0", "result": ["0.46.1", "0.0"], "id": 1}`. If you do, congratulations! You've set up your own wallet server.
To check Elastic search, there are two commands you can use:
```
curl localhost:9200 # get Elastic status
curl localhost:9200/claims/_count # check how many claims have been synced to Elastic
```
## Maintenance
### Stopping and Restarting
Use the usual docker-compose commands (`start`, `stop`, `pause`, etc) to control the servers. Run `docker-compose --help` to see the
options.
### Updating
To update to the latest wallet server release, run the following:
```
docker pull lbry/wallet-server:latest-release
docker-compose down
docker-compose up --detach
```
### Resyncing
From time to time, we'll release an update that requires recreating one of the databases from scratch. Most of the time we will try to ensure there is an automatic migration, but even then, if you think the server has invalid data you can also try a resync.
The process is similar to an update, but causes the server to be down for much longer.
#### Main database
Holds the raw blockchain data and takes several days to resync from scratch, so be sure to have a snapshot or try that last.
```
docker pull lbry/wallet-server:latest-release
docker-compose down
docker volume rm "$(whoami)_wallet_server"
WALLET_SERVER_SNAPSHOT_URL= docker-compose up --detach
```
#### Elasticsearch
ES does the indexing of claims from the main database. It should take around 6 hours to resync on a fast machine.
```
docker pull lbry/wallet-server:latest-release
docker-compose down
docker volume rm "$(whoami)_es01"
docker-compose up --detach
```
## Get in touch
Whether you got to the end without a hiccup or you got stuck along the way, we want to hear from you. [Join our Discord](https://discord.gg/y3W9JuS) to get help, stay updated, and talk to other wallet server operators.

79
src/content/resources/web-instance.md Normal file
View file

@ -0,0 +1,79 @@
---
title: Hosting your own LBRY Web Instance
description: Setting up an app instance as a webpage.
---
Run your own instance of https://lbry.tv using Docker images.
## Run the SDK
The LBRY SDK provides RPC and streaming endpoints to interact with the LBRY network. Web users will connect to it directly, so it must be web-accessible. You may have to open ports on your firewall.
```
docker run -d -p 5279:5279 -p 5280:5280 vshyba/websdk
```
This image will not save files to disk. It has the `save_blobs` and `save_files` config options set to `false`. If you want to save files, see [Building your own SDK image](#building-your-own-sdk-image) below.
## Run the web app
Clone and install the app as described in the [lbry-desktop repo README](https://github.com/lbryio/lbry-desktop).
If you want to customize it further, follow the extra steps in `Customize the web app` section. Otherwise:
```
git clone https://github.com/lbryio/lbry-desktop.git
yarn
cp .env.defaults .env
```
Configure .env with the following settings. They must match the SDK ports in the previous section.
```
WEB_SERVER_PORT=8080
SDK_API_PATH=http://localhost:5279
LBRY_WEB_API=http://localhost:5279
LBRY_WEB_STREAMING_API=http://localhost:5280
LBRY_API_URL=http://disabled-api/
LBRY_WEB_BUFFER_API=https://disabled
```
Compile and run
```
NODE_ENV=production yarn compile:web
nodejs web/index.js
```
## Building your own SDK image
If you want to customize the SDK settings, you can
Clone the SDK repo:
```
git clone https://github.com/lbryio/lbry-sdk.git
```
Create a `docker/webconf.yaml` file and modify as you need. This is a good start:
```
allowed_origin: "*"
max_key_fee: "0.0 USD"
save_files: false
save_blobs: false
streaming_server: "0.0.0.0:5280"
api: "0.0.0.0:5279"
data_dir: /tmp
download_dir: /tmp
wallet_dir: /tmp
```
Note that it is required to have `streaming_server` and `api` set to user-accessible IPs. If you want this to be accessible on the open web, that means setting them to `0.0.0.0`.
To build the image, run:
```
docker build -f docker/Dockerfile.web -t <your dockerhub username>/<project name, like 'websdk'> .
docker push <dockerhub username/project name>
```

77
src/content/tutorials/hellosatoshi.mdx Normal file
View file

@ -0,0 +1,77 @@
---
title: Hello Satoshi - The LBRY “Hello World” Tutorial
---
## Hello Satoshi - The LBRY "Hello World" Tutorial
Let's get started with a simple "Hellow World" tutorial... LBRY style!
This tutorial will guide you through creating a basic [Electron](https://electronjs.org) application that calls to the LBRY network and renders an image returned by the network.
Electron is nice because it allows you to easily create web apps that don't rely on any centralized web servers, but you can absolutely use any tooling or language you would like.
### Prerequisites
This tutoral only has a few simple requirements:
- [npm](https://www.npmjs.com). Learn how to install it [here](https://www.npmjs.com/get-npm).
- [git](https://git-scm.com/).
Once you have those installed (see the links above for downloads and How-To's), you are ready to begin!
#### Step 1. Download and build the starter project
Grab "[electron-starter](https://github.com/lbryio/electron-starter)". This project serves as a base upon which you can build LBRY applications. (Similar to "create-react-app" for React development.)
If you have git and npm installed, run the following lines one at a time:
```bash
git clone https://github.com/lbryio/electron-starter
cd electron-starter
npm install
npm run dev
```
#### Step 2. Make sure everything works
Before we make any changes, it's a good idea to verify that everything is working correctly.
Try typing a word into the text input and click the button to [resolve](https://lbry.tech/api/sdk#resolve) it.
This performs a [[claim]] lookup, which retrieves metadata the title, thumbnail, and file type from the LBRY blockchain.
Try resolving `lbry://doitlive`.
If you received no errors, move on to Step 3! Otherwise, head back to Step 1 to make sure you have all the requirements installed correctly.
#### Step 3. Make a small change to the code
Now that we have the metadata, let's [get](https://lbry.tech/api/sdk#get) the actual file!
The code to do this is already there, just un-comment these lines in the app's [renderer/index.js](https://github.com/lbryio/electron-starter/blob/master/src/renderer/index.js) file.
```js
claimData.innerText = "Loading...";
Lbry.get({ uri: `lbry://${value}` })
.then(result => {
const filePath = result.download_path;
const image = document.createElement("img");
image.src = filePath;
imageWrapper.appendChild(image);
claimData.innerText = JSON.stringify(result, null, 2);
})
.catch(error => {
claimData.innerText = JSON.stringify(error, null, 2);
});
```
This is the code that actually downloads a file.
There are more robust ways to handle the download progress, but this will work fine for images. After you added that code back, try `get`ing `lbry://doitlive`.
### Success! You Did It!
While our Hello Satoshi app isn't much to look at, it shows how simple it is to connect to the LBRY network and download files!

1
src/env.d.ts vendored
View file

@ -1 +1,2 @@
/// <reference path="../.astro/types.d.ts" />
/// <reference types="astro/client" /> /// <reference types="astro/client" />

1
src/layouts/Layout.astro
View file

@ -15,6 +15,7 @@ const { title } = Astro.props;
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">
<head> <head>
<meta charset="UTF-8">
<title>{title || 'LBRY Tech'}</title> <title>{title || 'LBRY Tech'}</title>
<meta name="apple-mobile-web-app-capable" content="yes"/> <meta name="apple-mobile-web-app-capable" content="yes"/>

34
src/layouts/Markdown.astro
View file

@ -1,12 +1,14 @@
--- ---
import Layout from './Layout.astro'; import Layout from './Layout.astro';
import { markdown } from '@astropub/md' import { markdown } from '@astropub/md';
import { getCollection } from "astro:content";
import TableOfContents from '../components/TableOfContents.astro';
import '../styles/markdown.css'; import '../styles/markdown.css';
// import {compile} from '@mdx-js/mdx'
// const compiled = await compile() const { frontmatter, headings, collection } = Astro.props;
const { frontmatter } = Astro.props; const items = await getCollection(collection) || [];
const description = await markdown(frontmatter.description) const description = await markdown(frontmatter.description)
@ -14,20 +16,40 @@ const description = await markdown(frontmatter.description)
<Layout title={frontmatter.title}> <Layout title={frontmatter.title}>
<div class="wrapper"> <div class="wrapper">
<section>
{items.length ? (
<a href={`/${collection}`}>{collection.charAt(0).toUpperCase() + collection.slice(1)}</a>
<ul>
{items.map(item=> (
<li><a href={`/${item.collection}/${item.slug}`}>{item.data.title}</a></li>
))}
</ul>
) : null}
</section>
<div class="markdown-body"> <div class="markdown-body">
<h1>{frontmatter.title}</h1> <h1>{frontmatter.title}</h1>
<h3>{description}</h3> <h3>{description}</h3>
<slot/> <slot/>
</div> </div>
<section class="toc">
<TableOfContents headings={headings} />
</section>
</div> </div>
<style> <style>
.wrapper { .wrapper {
margin: auto; display: flex;
max-width: 1000px; margin: 20px;
/* max-width: 1000px; */
} }
.wrapper .markdown-body { .wrapper .markdown-body {
margin: 10px 20px; margin: 10px 20px;
max-width: 1000px;
}
.wrapper .toc {
display: flex;
flex-direction: column;
} }
</style> </style>
</Layout> </Layout>

12
src/pages/resources.astro Normal file
View file

@ -0,0 +1,12 @@
---
import Markdown from '../layouts/Markdown.astro';
const frontmatter = {
title: "Find the LBRY specification, API documentation, our Contributor’s guide, and more in the Resources area.",
description: "Find the LBRY specification, API documentation, our Contributor's guide, and more in the Resources area."
}
---
<Markdown title="Resources" frontmatter={frontmatter} collection="resources">
</Markdown>

20
src/pages/resources.mdx
View file

@ -1,20 +0,0 @@
---
layout: '../layouts/Markdown.astro'
title: "Resources"
description: "Find the LBRY specification, API documentation, our Contributor's guide, and more in the Resources area."
---
## Additional Resources
- [Developer Setup Intro Videos](/resources/setup-videos)
- [LBRY Glossary](/glossary)
- [LBRY Merkle Claim Trie](/resources/claimtrie)
- [LBRY Consensus Algorithm](/resources/consensus)
- [Download Overview](/resources/download-overview)
- [API Wrappers](/resources/api-wrappers)
- [LBRY SDK Configuration Settings](/resources/daemon-settings)
- [Claim Signing](/resources/claim-signing)
- [LBRY Android App Build Steps](/resources/android-build)
- [Lighthouse (search) API](https://github.com/lbryio/lighthouse)
- [Run Your Own Wallet Server](/resources/wallet-server)
- [Run Your Own lbry.tv](/resources/web-instance)

8
src/pages/resources/[slug].astro
View file

@ -3,16 +3,16 @@ import { getCollection } from 'astro:content';
import Markdown from '../../layouts/Markdown.astro'; import Markdown from '../../layouts/Markdown.astro';
export async function getStaticPaths() { export async function getStaticPaths() {
const blogEntries = await getCollection('resources'); const entries = await getCollection('resources');
return blogEntries.map(entry => ({ return entries.map(entry => ({
params: { slug: entry.slug }, props: { entry }, params: { slug: entry.slug }, props: { entry },
})); }));
} }
const { entry } = Astro.props; const { entry } = Astro.props;
const { Content } = await entry.render(); const { Content, headings } = await entry.render();
--- ---
<Markdown frontmatter={entry.data}> <Markdown frontmatter={entry.data} headings={headings} collection={entry.collection}>
<Content /> <Content />
</Markdown> </Markdown>

31
src/pages/tutorials.astro Normal file
View file

@ -0,0 +1,31 @@
---
import Markdown from '../layouts/Markdown.astro';
const frontmatter = {
description: "Find the LBRY specification, API documentation, our Contributor's guide, and more in the Resources area."
}
---
<Markdown title="Tutorials" frontmatter={frontmatter} collection="tutorials">
<h2>Setup your Development Environment</h2>
<h3>Desktop Application</h3>
<p><a href="/resources/video-lbrydesktop">Video tutorial</a> to setup the <a href="https://github.com/lbryio/lbry-desktop">Desktop app</a> development environment.</p>
<h3>Android Application</h3>
<p><a href="/resources/video-lbryandroid">Video tutorial</a> to setup the <a href="https://github.com/lbryio/lbry-android">Android app</a> development environment.</p>
<h3>LBRY SDK</h3>
<p><a href="/resources/video-lbrysdk">Video tutorial</a> to setup the <a href="https://github.com/lbryio/lbry-sdk">LBRY SDK</a> development environment.</p>
<h3>LBRY Blockchain</h3>
<p><a href="/resources/video-lbrycrd">Video tutorial</a> to setup the <a href="https://github.com/lbryio/lbrycrd">LBRY Blockchain</a> development environment.</p>
</Markdown>
---
layout: '../layouts/Markdown.astro'
title: Tutorials
description: Learn how to setup, use, deploy, and develop with LBRY.
---

26
src/pages/tutorials.mdx
View file

@ -1,26 +0,0 @@
---
layout: '../layouts/Markdown.astro'
title: Tutorials
description: Learn how to setup, use, deploy, and develop with LBRY.
---
# LBRY Programming Tutorials
## Tutorial #1 - "Hello Satoshi!"
Learn how to [create and modify a simple LBRY electron application](/tutorial-hellosatoshi) we'll call "[Hello Satoshi](/tutorial-hellosatoshi)".
---
# Setup your Development Environment
## Desktop Application
[Video tutorial](/resources/video-lbrydesktop) to setup the [Desktop app](https://github.com/lbryio/lbry-desktop) development environment.
## Android Application
[Video tutorial](/resources/video-lbryandroid) to setup the [Android app](https://github.com/lbryio/lbry-android) development environment.
## LBRY SDK
[Video tutorial](/resources/video-lbrysdk) to setup the [LBRY SDK](https://github.com/lbryio/lbry-sdk) development environment.
## LBRY Blockchain
[Video tutorial](/resources/video-lbrycrd) to setup the [LBRY Blockchain](https://github.com/lbryio/lbrycrd) development environment.

18
src/pages/tutorials/[slug].astro Normal file
View file

@ -0,0 +1,18 @@
---
import { getCollection } from 'astro:content';
import Markdown from '../../layouts/Markdown.astro';
export async function getStaticPaths() {
const entries = await getCollection('tutorials');
return entries.map(entry => ({
params: { slug: entry.slug }, props: { entry },
}));
}
const { entry } = Astro.props;
const { Content, headings } = await entry.render();
---
<Markdown frontmatter={entry.data} headings={headings} collection={entry.collection}>
<Content />
</Markdown>