This commit is contained in:
Thomas Zarebczan 2019-09-10 15:43:28 -04:00 committed by GitHub
parent d7f20e6827
commit 1cb87f3b1b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -12,9 +12,21 @@ To read previous updates, please visit our [Development and Community Update arc
If you want to see a condensed view of what we have completed recently and whats planned for LBRY, check out our [Roadmap](https://lbry.io/roadmap).
First off, lets start with some GitHub stats across all our repos (some of which are internal only to LBRY) since our last update (~40 days):
36 repos were updated. 617 issues were created, 146 were closed. 18 PRs were merged. 30 GitHub users made code contributions: AlessandroSpallina, Borewit, BrannonKing, Feanor78, NetOperatorWibby, akinwale, btzr-io, bvbfan, ceoger, dittaeva, eggplantbren, eukreign, finer9, jackrobison, jessopb, jsigwart, kauffj, lbrynaut, lyoshenka, nestordominguez, nikooo777, osilkin98, sayplastic, seanyesmunt, shyba, tiger5226, tzarebczan, vv181, ykris45, zxawry.
36 repos were updated. 617 issues were created, 146 were closed. 18 PRs were merged. 30 GitHub users, outside of LBRY, made code contributions:
- [AlessandroSpallina](https://github.com/AlessandroSpallina)
- [Borewit](https://github.com/Borewit)
- [btzr-io](https://github.com/btzr-io)
- [ceoger](https://github.com/ceoger)
- [dittaeva](https://github.com/dittaeva)
- [eggplantbren](https://github.com/eggplantbren)
- [nestordominguez](https://github.com/nestordominguez)
- [vv181](https://github.com/vv181)
- [ykris45](https://github.com/ykris45)
- [zxawry](https://github.com/zxawry)
Thanks to everyone who took time out of their busy days to help LBRY out!
# In this report, we cover updates for {#dev-updates}
## In this report, we cover updates for
* [Desktop App](#summary-desktop)
* [SDK](#summary-sdk)
* [Blockchain](#blockchain)
@ -23,18 +35,32 @@ First off, lets start with some GitHub stats across all our repos (some of wh
* [Rewards](#rewards)
* [Cross Device Syncing](#cross)
* [YouTube Sync Transfers](#youtube)
* [LBRY Reflector](#reflector)
* [Community Block Explorer](#explorer)
* [2019 Roadmap Progress](#roadmap)
### Desktop App {#summary-desktop}
Since our last development update, the app team shipped our next named release, 0.35, under codename [Franklin](https://lbry.com/news/franklin-is-minted) as well as a few patches. Franklin enhances the app experience with a brand new videojs-based video player. This will allow us to customize more aspects of the player in the future.
Since our last development update, the app team shipped our next named release, 0.35, under codename [Franklin](https://lbry.com/news/franklin-is-minted) as well as a few patches. Franklin enhances the app experience with a brand new [videojs-based](https://github.com/videojs/video.js) video player.
Other new features include the ability to continue watching videos in pop out mode while checking out other content in the app, support of range requests (allows streaming mode), and the ability to block channels app wide.
Other new features in Franklin include:
- the ability to continue watching videos in pop out mod
- support of range requests (streaming)
- the ability to block channels app wide
The patch releases include three main new features: tipping on channel pages (which gets sent to the channel claim vs content claims on file pages), ability to set the time period for dark mode, and support for a new daily watch reward that will be live in mid-September (replacing the weekly LBRYCast).
The patch releases included some new features:
- all new features added to the SDK between version 0.38.5 and 0.40, including improved wallet sync behavior, download speeds, and overall connectivity ([more details below](#summary-sdk))
- tipping on channel pages (which gets sent to the channel claim vs content claims on file pages)
- ability to set the time period for dark mode
- improvements to the display of URLs (now has the [canonical URL](https://lbry.tech/glossary#canonical-url) which uses both channel and claim short IDs)
- support for a new daily watch reward that went live on September 10th (replacing the weekly LBRYCast)
The patches also include bug fixes to bid amounts required on the publish page (did not show vanity URL values), auto downloading subscription content, and a few other smaller issues, as well as changes to the URL format (now has both channel and claim short ids), updating our digital certificate, and getting the SDK updated to the latest release ([0.40.0](https://github.com/lbryio/lbry-sdk/releases/tag/v0.40.0) for improved wallet sync behavior, download speeds, and overall connectivity.
The patches fixed many bugs, including:
- issues with bid amounts required on the publish page (did not show vanity URL bids),
- auto downloading subscription content
- search bar focus getting stuck
- disabling autoplay on paid content
- DMCA blocking to include entire channels
We also updated our digital certificate for Windows builds which is a yearly maintenance task. We'll explore a better solution for this down the road.
![franklin](https://spee.ch/9/35.gif)
@ -54,30 +80,39 @@ Another area to continue getting additional attention is internationalization. Y
### SDK {#summary-sdk}
On the SDK side of the house, weve shipped versions 0.39, with patches, and 0.40.0 with a recent patch. You can review all the changelogs on our [GitHub releases page](https://github.com/lbryio/lbry-sdk/releases).
The main features include treating multiple accounts inside a wallet to effectively be treated as one (missing a few API calls, to be completed soon), ability to connect and synchronize transactions with multiple wallet servers (see screenshot below) which improves wallet sync speed and paves the way to interfacing with community wallet servers, wallet server monitoring tools to include session information / logging of troublesome claim search queries, and a key value store which will be the final location of synced LBRY data. On startup, we also ping all the wallet servers and select the best main one to connect to based on the peers location. We also added a new function for comments that allows channel owners to moderate comments by hiding them (to be implemented in the app in the next few months). The ability to connect/sync with multiple servers proved to be more troublesome than expected, we experienced and fixed reconnection issues which should be resolved with the [0.40.1 patch](https://github.com/lbryio/lbry-sdk/releases/tag/v0.40.1).
The main features include:
- treating multiple accounts inside a wallet to effectively be treated as one (missing a few API calls, to be completed soon)
- ability to connect and synchronize transactions with multiple wallet servers (see screenshot below) which improves wallet sync speed and paves the way to interfacing with community wallet servers
- wallet server monitoring tools to include session information / logging of troublesome claim search queries
- a key value store which will be the final location of synced LBRY data.
- added a new function for comments that allows channel owners to moderate comments by hiding them (to be implemented in the app in the next few months)
On SDK startup, we now ping all the wallet servers and select the best main one to connect to based on the peers location. The ability to connect/sync with multiple servers proved to be more troublesome than expected, we experienced and fixed reconnection issues which should be resolved with the [0.40.1 patch](https://github.com/lbryio/lbry-sdk/releases/tag/v0.40.1).
![settings and wallet sync](https://spee.ch/8/sdk-settings.png)
#### SDK Next Steps
During testing of the [YouTube transfer](#youtube) process, we ran into a snag with how our channel import/export process was designed, especially when the channel itself is transferred from one wallet to another which prevented both sides (YouTube Sync and the Content Creator) from signing claims. Progress has been made to support a short term solution for both cases, and we expect a resolution soon. Besides this bug, we are also looking into any remaining wallet syncing / connectivity problems, one affecting content announcement, and another that may cause memory leaks on some machines. If you think your node has suffered from either, please give us a shout on [Discord](https://chat.lbry.com).
Finally, weve also been researching and began initial implementation of a technique called holepunching, which will improve connectivity between nodes -- especially for those behind firewalls and atypical network configurations. This may alleviate the need for port forwarding, but nodes that can do port forwarding will be encouraged to leave it on. We expect this to increase our overall network availability and performance.
Finally, weve also been researching and began initial implementation of a technique called [holepunching](https://en.wikipedia.org/wiki/TCP_hole_punching), which will improve connectivity between nodes -- especially for those behind firewalls and atypical network configurations. This may alleviate the need for port forwarding, but nodes that can do port forwarding will be encouraged to leave it on. We expect this to increase our overall network availability and performance.
### Blockchain {#blockchain}
Since our last update, the blockchain team continues their work towards enabling SegWit later this year, optimizing memory usage, revamping RPC calls to include bid (allow us to show pending effective amounts during takeovers) and sequence (which was removed in a previous update), and enabling metadata along with claims supports. Support metadata will enable us to expand the support/tipping feature of LBRY, which currently only allows simple LBC transactions to occur, to include additional metadata that can be used for signing the supports to prove channel ownership and enabling use cases like commenting, rating, and subscriptions on chain. Signing of supports will finally give us the ability to link a channel back to a tip transaction, so publishers can see and verify who sent LBC (or even a support). On chain subscriptions also help further decentralize data you can only find on our API currently, and enable a Patreon like system where creators can release content thats accessible only by their supportors or at various support/tip levels.
#### Forced Upgrade - Pre 0.17.2 Nodes Go Offline
On August 16, 2019, the LBRY chain experienced an unexpected fork between the current version of lbrycrd (v0.17.2) and previous versions. Block height of the fork: 617743. We might have favored the older build, but older versions of lbrycrd would crash when receiving data from the newer version after that block. v0.17.1 was able to proceed down its own chain, but there was little mining power behind this; the main pools were already mining with majority of hash on the correct (v0.17.2) chain. Everything is back to normal currently on the latest release/chain, and well continue to monitor. If you are still running a pre 0.17.2 node, please delete local
On August 16, 2019, the LBRY chain experienced an unexpected fork between the current version of lbrycrd (v0.17.2) and previous versions. The fork occured at block height 617743. Everything is back to normal currently on the latest release/chain, and well continue to monitor. If you are still running a pre 0.17.2 node, please [see below](#recover) on how to recover.
Read [our blog post](https://lbry.com/news/lbry-blockchain-update) for the entire executive summary of the incident.
#### How to Recover a Pre 0.17.2 Node
#### How to Recover a Pre 0.17.2 Node {#recover}
Please upgrade your LBRYcrd node to the latest version on our [GitHub release page](https://github.com/lbryio/lbrycrd/releases/tag/v0.17.2.1). After youve run it, perform the following CLI command: `lbrycrd-cli reconsiderblock b9a350fa8b4471c46cb2088927ac5e959939c815e7d98e8125f099b902e96a62` to get back up to speed. The other option would be to clear our your local data and resync from scratch.
### LBRY.tv {#web}
Over the last few months, weve been running a view only pilot of [beta.lbry.tv](https://beta.lbry.tv) meaning you can view free content, but many of the other features like wallets, rewards, and publishing have been disabled. Weve also created/ran stress tests on lbry.tv to make sure the web server and underlying SDK can handle thousands of concurrent users. In order to scale to even more users in the future, we will need to run multiple web and SDK servers.
Recently we introduced application options on the Settings page, but none of these options persist between refreshes until were able to sync the settings to our interim API sync solution. Longer term, these will sync back to the wallet file once lbry.tv migrates from an account based model, to a wallet based one that will support cross device syncing (today, syncing only works at a wallet level). The current SDK does not support multiple wallets, which is why we went with an account based model (to at least ship a full featured web experience without wallet sync). Weve also recently added OpenGraph metadata to channels and claim links when shared on Twitter/Facebook and other sites that support it.
Recently we introduced application options on the Settings page, but none of these options persist between refreshes until were able to sync the settings to our interim API sync solution. Longer term, these will sync back to the wallet file once lbry.tv migrates from an account based model, to a wallet based one that will support cross device syncing (today, syncing only works at a wallet level). The current SDK does not support multiple wallets, which is why we went with an account based model (to at least ship a full featured web experience without wallet sync).
We also added OpenGraph metadata to channels and claim links when shared on Twitter/Facebook and other sites that support it.
![metadata](https://spee.ch/c/tv-metadata.png)
@ -101,7 +136,7 @@ Want to be the first to know when its fully operational? Sign up for the mail
Since our last update, the Android app has been playing catchup with the Desktop apps discovery features - this includes the ability to follow tags and explore tags, create channels, and publish (and edit!) content. Were also excited that some of the more profound performance problems with the Android app have been resolved under the hood - the latest release provides a much smoother experience navigating throughout the app. The publish process was streamlined for mobile purposes which includes automatic uploading of thumbnails. You can also tag your content, create a channel, select a URL, and adjust your default LBC bid. On another note, we have noticed some issues with syncing wallets on restore, so if you run into this, please reach out to us on [Discord](https://chat.lbry.com) - we are still investigating but this will be solved once the SDK fully supports multiple accounts.
#### Discovery
The discovery process, which resembles the Desktop application, allows you to customize your followed tags which are then shown on the `Your Tags` page - this includes a drop down to change between trending/top/new, along with a top level listing of content across all your tags, plus a break down for each tag you follow with an option to view the tag page (click the arrows, or More tile). Currently we have to limit this to 5 tags (randomized between runs) which are shown on the main page due to performance issues, but you can click on the rest of your tags at the bottom of the page to see their tag pages. The subscriptions area also received a revamp - you can now see Trending/Top/New views across all your subscriptions, or you can click on a specific creator to filter. The `All Content` page enables you to explore other LBRY content not in your tags (default setting) using the same Trending/Top/New sorting options (you can also choose to limit on tags you follow which is the view youd get to by clicking More on the top item from the `Your Tags` homepage.
The discovery process, which resembles the Desktop application, allows you to customize your followed tags which are then shown on the "Your Tags" page - this includes a drop down to change between trending/top/new, along with a top level listing of content across all your tags, plus a break down for each tag you follow with an option to view the tag page (click the arrows, or More tile). Currently we have to limit this to 5 tags (randomized between runs) which are shown on the main page due to performance issues, but you can click on the rest of your tags at the bottom of the page to see their tag pages. The subscriptions area also received a revamp - you can now see Trending/Top/New views across all your subscriptions, or you can click on a specific creator to filter. The "All Content" page enables you to explore other LBRY content not in your tags (default setting) using the same Trending/Top/New sorting options (you can also choose to limit on tags you follow which is the view youd get to by clicking More on the top item from the "Your Tags" homepage.
![android](https://spee.ch/@lbry:3f/android-08-homepage.gif)
@ -126,12 +161,7 @@ During the last 2 months, the team has worked on bringing creators closer to the
This feature is a culmination of efforts across multiple teams at LBRY:
The SDK team was responsible for exposing the necessary tools to export and update channels and streams, the Backend team was responsible for exposing the endpoints necessary to initiate and pursue the process, the App team had to interface with such endpoints and last but not least, the YouTube sync team had to ensure the integrity of each channel before transferring everything to the creator.
The result is pretty fascinating and closes the last loop in the sync process: Youtube creators can click a single button in the app, immediately gain access to publish new content via their channel, and their wallet will soon be populated with all their content and channel claims, including all their tips being re-supported on their channel. Were super excited for this one team wide!
### LBRY Reflector {#reflector}
The Reflector server run by LBRY is part of our infrastructure allows the LBRY network to stay healthy in this bootstrap phase where data availability through the P2P network might not be as satisfactory as expected. The technology behind this software spans across many layers and allows virtually all the content ever published to LBRY to be available for download at high speeds. To date we host 240TB (240,000GB) of blobs making up for over a million different claims.In order to keep track of all these blobs we manage a pretty big database.
After a recent increase in volume of published content we had to optimize such infrastructure to contain costs and improve scalability. Starting from mid August we're rocking a fresh new database that performs/scales better and uses less space than the previous one. Over the next few months, well be making more improvements so that anyone can run one of these servers and become a larger host on the LBRY network.
The result is pretty fascinating and closes the last loop in the sync process: YouTube creators can click a single button in the app, immediately gain access to publish new content via their channel, and their wallet will soon be populated with all their content and channel claims, including all their tips being re-supported on their channel. Were super excited for this one team wide!
### Community Block Explorer {#explorer}
A community member by the name of [Allesandro Spallina](https://github.com/AlessandroSpallina) became interested in the LBRY ecosystem and wanted to implement a more modern and full featured block explorer that exposes important metrics of the LBRY blockchain. You can check out his [GitHub issue](https://github.com/lbryio/block-explorer/issues/67) outlining what he plans on developing or you can go straight to the explorer prototype at [lbryexplorer.tech](https://lbryexplorer.tech/) to give it a shot Wed appreciate any feedback on the GitHub issue and current prototype - you can leave a comment on GitHub or reach out on [Discord](https://chat.lbry.com) #dev channel by tagging @SK3LA.
@ -155,8 +185,6 @@ If you arent part of our Discord community yet, [join us](https://chat.lbry.c
Our community allows LBRYians to interact with the team directly and for us to engage users in order to grow the LBRY platform. Also follow us on [Twitter](https://twitter.com/lbryio), [Facebook](https://facebook.com/lbryio), [Reddit](https://www.reddit.com/r/lbry), [BitcoinTalk](https://bitcointalk.org/index.php?topic=5116826.new#new), and [Telegram](https//t.me/lbryofficial).
[Back to **top**](#dev-updates)
Weve got a special bonus for readers of this update, enjoy some LBC via this code (while supplies last!): `dev-update-aug-jkala`
Thanks for supporting LBRY - stay tuned for more news and updates! And if you havent downloaded the [LBRY app](https://lbry.io/get?auto=1) yet, what are you waiting for?